Data Engineer

DWP. Digital with Purpose.

We need a Data Engineer to join our Management Information (MI) team and help shape the future of data king with modern AWS technologies, you'll develop scalable data solutions that power reporting, analytics, and strategic decision-making.

Our DWP Digital teams use fresh ideas and leading-edge tech to build and maintain digital solutions that will be used by nearly every person in the UK, every day and at key moments in their lives.

DWP is the UK's largest government department. We help people into work and make payments worth over £195bn a year to support and empower millions of people.

The scale of what we do is extraordinary, and our purpose is unique. We'd love you to join us.

What skills, knowledge and experience will you need?

• Experience developing and managing PySpark workflows within AWS environments.
• Knowledge of AWS Lambda and event-driven solutions to support automated data processing.
• Experience provisioning and supporting cloud infrastructure using Amazon EC2 and automation tools.
• Understanding of Agile ways of working and experience contributing to CI/CD delivery practices.
• Ability to optimise data processing, storage, and queries to improve performance and data quality.
• Experience designing and automating ETL pipelines using PySpark and Spark SQL...