Senior Data Engineer

On behalf of The Ministry of Justice, we are looking for a Senior Data Engineer (Inside IR35) for a 6 month contract based primarily remote (occasional travel to MoJ offices and team meetings, typically in London).The Synergy Programme is a large-scale transformation programme consisting of four Government departments, MOJ (Ministry of Justice), DWP (Department for Work and Pensions), DEFRA (Department for Environment, Food and Rural Affairs) and HO (Home Office).

They will come together to replace their current back-office systems and introduce new ways of working for 220,000 civil servants combined. High-quality, accurate data is fundamental to ensuring a successful transition and enabling effective business processes, reporting and operations within the new ERP platform.

The Data Engineer will play a critical role in supporting the MoJ contribution to the Synergy Programme by providing the technical capability required to extract, transform, manage and prepare source data for analysis. This role directly supports key programme priorities, including data quality improvement and data migration readiness.

This role requires strong hands-on AWS Data Engineering experience, with a particular focus on building and maintaining automated data pipelines and cloud-based data solutions using AWS services, Python and SQL.As a Senior Data Engineer, your main responsibilities will include:
* Designing, building and maintaining robust, scalable and automated data pipelines that extract source data from SplashBI and load it into the programme's AWS data platform.
* Working closely with Data Analysts to understand user needs and data requirements, translating them into technical requirements and solutions.
* Identifying opportunities to improve, automate, reuse and re-engineer data engineering processes to increase efficiency, enable scaling and reduce manual effort.
* Maintaining secure and sustainable AWS cloud infrastructure using infrastructure-as-code principles and best practice development approaches, deploying solutions using a range of AWS services in a reproducible and sustainable manner.
* Developing and optimising code, tools and services that enable analysts to access and use data effectively and efficiently.
* Proactively monitor, maintain and optimise data pipelines to ensure source data is available to analysts in line with programme timelines and migration readiness milestones.
* Operate effectively in a fast-paced programme environment, managing competing priorities and delivering technical solutions to demanding timescales.

Essential:
* Strong hands on AWS Data Engineering experience, including developing, maintaining and optimising robust and scalable automated data pipelines using Python, SQL and AWS cloud services.
* Experience working in a technical team, preferably one that uses Python and SQL, as well as strong technical skills in these languages.
* Working with a range of data sources and source systems and integrating data for analysis and migration purposes including data wrangling and analysis activities.
* Understanding of the software development lifecycle and applying software development best practices, including version control (e.g. git), automated testing (e.g. pytest) and CI/CD (e.g. Git Hub Actions).
* Experience using infrastructure-as-code approaches to deploy and manage AWS cloud infrastructure.
* Supporting data quality, migration or business intelligence activities through effective data engineering practices.
* Working with technical and non-technical stakeholders to understand requirements and deliver effective data solutions.
* Experience working in a fast-paced delivery environment, managing multiple priorities and delivering high-quality outputs to challenging…