Description
Location:
- West Midlands (4 or 5 days on client site)
Clearance Requirements:
- SC clearance required or willingness to undergo the SC process
- Sole UK Nationals only
Role Overview:
We are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems that operate across both on-premises and Azure cloud environments. The position involves deploying and managing scalable data operations that support advanced analytics and data-driven decision-making, crucial for our organisational growth and innovation.
Key Responsibilities:
- Develop and Manage Data Pipelines: Design, construct, and maintain efficient and reliable data pipelines using Python, capable of supporting both streaming and batch data processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
- Hybrid Cloud and Data Storage Solutions: Implement and manage data storage solutions leveraging both on-premises infrastructure and Azure, ensuring seamless data integration and accessibility across platforms.
- Containerisation and Orchestration: Utilise Docker for containerisation and Kubernetes for orchestration, ensuring scalable and efficient deployment of applications across both cloud-based and on-premises environments.
- Workflow Automation: Employ tools such as Apache NiFiand Apache Airflow to automate data flowsand manage complex workflows within hybrid environments.
Additional Information:
This role will require you to have or be willing to go through Security Clearance. As part of the onboarding process candidates will be asked to complete a Baseline Personnel Security Standard; details of the evidence required to apply may be found on the government website Gov.UK. If you are unable to meet thisand any associated criteria then your employment may be delayed or rejected.Details of this will be discussed with you at interview