Data Engineer SC Cleared - AWS - IR35

Position: Data Engineer – SC Cleared - AWS - Inside IR35
Location: Farringdon

Data Engineer – SC Cleared - AWS - Inside IR35
We are supporting a major government data transformation initiative to strengthen the use of evidence-based insights across frontline and operational teams. As part of a new capability being built to process and analyse sensitive interview information, the programme requires Data Engineers to design, deliver, and optimise secure backend data workflows.
This work is foundational: building the ingestion, orchestration, storage, and transformation layers that power the analytics tool.

Key Responsibilities

Design, develop and maintain scalable cloud-native data pipelines

• Implement ETL/ELT processes to manage structured and unstructured data securely and efficiently

• Ensure data integrity, traceability and compliance across all pipeline stages

• Work with cross-functional teams to define technical requirements and design decisions

• Apply Dev Ops best practices, monitoring, and automation to improve reliability

• Support continuous improvement of the platform’s performance and operational maturity

• Communicate progress, risks and trade-offs clearly to wider delivery stakeholders

Required Skills & Experience

Strong Data Engineering expertise within AWS environments

• Hands-on experience with core AWS data services:
 – S3, Glue, Lambda, Athena, Kinesis, Step Functions (or similar)

• Proficiency in Python and SQL for data transformations and automation

• Experience with IaC and CI/CD tooling (Terraform, Git Lab, etc.)

• Comfortable working with sensitive datasets and secure-by-design approaches

• Strong communication skills and a proactive, consulting mindset

• Experience delivering as part of an Agile, multi-disciplinary team Desirable

Knowledge of backend processing for analytics workloads (but not essential)

• Familiarity with containerised deployments (Docker/ECS)

• Experience working to SFIA-aligned delivery expectations in government or regulated contexts