Outside IR35 contract jobs

Description

We are seeking an experienced Data Engineer with a strong background in building and optimizing data pipelines, managing data infrastructure, and enabling data-driven decision-making. The ideal candidate will have at least 7-8 years of experience in data engineering and a proven track record of working with modern data technologies and tools. This role requires a self-starter who can work independently within a team, handle tight deadlines, and contribute to the success of our data initiatives.

Key Responsibilities
- Data Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines using Python, Spark, and PySpark to process large volumes of structured and unstructured data.
- Data Warehousing: Build and optimize data marts and warehouses in Snowflake to support analytics and reporting needs.
- Workflow Automation: Implement and manage workflows using Airflow for scheduling and orchestrating data pipelines.
- Cloud Infrastructure: Leverage AWS services (eg, EMR, S3, Lakehouse architecture) to build and maintain cloud-based data platforms.
- CI/CD Integration: Utilize GitLab CI/CD pipelines to automate testing, deployment, and monitoring of data solutions.
- Data Quality Management: Ensure high-quality data by implementing robust validation, monitoring, and governance frameworks.
- Business Collaboration: Work closely with stakeholders in the credit card and retail domains to understand business requirements and translate them into technical solutions.
- Performance Optimization: Continuously monitor and optimize the performance of data pipelinesand infrastructure.
- Documentation & Best Practices: Maintain comprehensive documentation for all processesand advocate for industry best practices indata engineering.

Required Skills
Proficiency in Python for scriptingand automation.
Expertise in Sparkand PySparkfor big dataprocessing.
Hands-on experience with Airflow/other toolsfor workflow orchestration.
Data Platforms:
Advanced knowledgeof Snowflakefordata warehousing.
Strong ETL/ELT skillswith experiencebuildingdatamarts.
Cloud Services:
Experiencewith AWSservices such as EMR,S3,and Lakehousearchitecture.
Version Control & CI/CD:
Familiaritywith GitLabCI/CDpipelinesfor continuousintegrationand deployment.
Data Quality & Management:
Knowledgeofdata qualityframeworks,valida tion techniques,and governancepractices.
Domain Expertise:
Businessexperiencein creditcardoperationsor retailindustriesis highlydesirable.

Preferred Qualifications
Strong problem-solving skillswiththe abilityto workindependentlywithinacollaborativeteam environment.
Proven abilityto deliverresultsunder tightdeadlineswhilemaintaininghigh-qualitystandards.
Excellent communication skillsforcross-functional collaborationwithtechnicalandnon-technica lstakeholders.

This is an exciting opportunityto joinadynamicteamwhereyoucanmakeasignificantimpactbyleveragingyourexpertiseinmoderndataengineeringtechnologies.If you thriveinfast-pacedenvironmentsandarepassionateaboutbuildingrobustdatasolutions,we encourageyoutoapply!