Description
Role: Senior Data DevOps Engineer
Location: London
Duration: 12-24 months
Rate: £650-900/day
Job Description
You will join a team of 6/7 Data DevOps Engineers operating in a central team that supports a number of data engineering teams. The data engineering teams are working with a very modern tech stack that includes AWS, Glue, Athena, Airflow, PySpark, Spark, Dremio, DBT, Iceberg, and Apache Arrow.
Key Responsibilities
- Collaborate as part of an Agile technology team to design, develop, troubleshoot, and solve complex software issues.
- Set up automated processes for the data engineers.
- Work closely with data engineering teams to deliver solutions.
- Maintain and enhance the tech stack to meet project-specific needs.
- Automate processes using CI/CD DevOps tools.
- Ensure the reliability, security, and performance of the infrastructure.
Essential Requirements
- Experience as a Senior DevOps Engineer/SRE/Python Developer in an Agile environment.
- Strong experience delivering solutions in the AWS (Amazon Web Services) cloud.
- Advanced ability in CI/CD DevOps tools.
- Good experience with open-source Kubernetes, including lower-level engineering and tweaks.
- Experience working with data engineering teams.
- Proficiency in Python.
- Experience with 2 or more of the following:
- Kafka
- Airflow
- Spark
- PySpark
- Dremio
- Ignite
- DBT
- Iceberg
- Arrow
- Glue
- Athena
Desirable Skills
- Java (J2EE or Core Java)
- Golang
Additional Information
- The role offers a very flexible working from home policy.
- You will be part of a central team supporting multiple data engineering teams with a modern tech stack.
- The position requires strong technical expertise and the ability to work collaboratively in an Agile environment.
Working Environment
- Flexible working from home policy.
- Central team supporting multiple data engineering teams.
- Modern tech stack including AWS, Glue, Athena, Airflow, PySpark, Spark, Dremio, DBT, Iceberg, and Apache Arrow.