Insight International (UK) Ltd

AWS Databricks Tech Lead
Apply Now

๐Ÿ“… Date Posted

Feb 24, 2025

๐Ÿ’ผ Job Type

CONTRACTOR

๐Ÿ“ Location

United Kingdom

๐Ÿ’ต Rate

Unknown

Description

Role: AWS Databricks Tech Lead
Location: Remote
Job Type: Contract

Key Skills:
- AWS Data / ETL work Experience -> S3, Glue, Athena
- Databricks work experience -> Data processing inside Databricks
- Data or ETL migration/Modernization experience (MUST for Architect, Optional for Lead but itโ€™s very much preferred)
- Proficient in SQL queries

The key is to get candidates who have worked in Databricks on AWS.

Experience and Education Required:
- 10+ years of experience as Data Analyst / Data Engineer/Data Scientist with Databricks on AWS expertise in designing and implementing scalable, secure, and cost-efficient data solutions on AWS.

Job Profile:
- Hands-on data analytics experience with Databricks on AWS, Pyspark and Python.
- Must have prior experience with migrating a data asset to the cloud using a GenAI automation option.
- Experience in migrating data from on-premises to AWS.
- Expertise in developing data models, delivering data-driven insights for business solutions.
- Experience in pretraining, fine-tuning, augmenting and optimizing large language models (LLMs).
- Experience in Designing and implementing database solutions, developing PySpark applications to extract, transform, and aggregate data, generating insights.

Data Collection & Integration:
- Identify, gather, and consolidate data from diverse sources including internal databases and spreadsheets ensuring data integrity and relevance.

Data Cleaning & Transformation:
- Apply thorough data quality checks cleaning processes and transformations using Python (Pandas) and SQL to prepare datasets.

Automation & Scalability:
- Develop and maintain scripts that automate repetitive data preparation tasks.

Autonomy & Proactivity:
- Operate with minimal supervision demonstrating initiative in problem-solving prioritizing tasks continuously improving the quality and impact of your work.

Technical Skills:
- Minimum of 10 years of experience as a Data Analyst Data Engineer or related role ideally with a bachelorโ€™s degree or higher in a relevant field.
- Strong proficiency in Python (Pandas Scikit-learn Matplotlib) SQL with experience working across various data formats sources.
- Proven ability to automate data workflows implement code-based best practices maintain documentation to ensure reproducibility scalability.

Share:
Interested in this job?
Apply Now

Subscribe Newsletter

Sign up to our newsletter to get Outside IR35 jobs directly to your inbox.