Data Engineer

We are transforming DCMS to meet the demands of an agile, delivery-focused department leading on some of the highest priority work for the government's agenda to drive growth, enrich lives and level-up society. Our modernisation plans include critical investment in our systems, ensuring they are fit for purpose and reducing the manual input required, as well as the ongoing development of a new central function to transform the way DCMS uses technology, data and digital services.

We also want to transform how we solve policy challenges by using user centred design and collaboration with using multi-disciplinary teams. This is an incredibly exciting time to join the team within DCMS. Our mission is to digitally enable how DCMS delivers its priority outcomes which can adapt and respond to the changing needs of the sectors and partners we work with.

Can you help us get there? You will work within an agile multidisciplinary team supporting the design, development and maintenance of data pipelines and systems on our Databricks platform. You will help ensure that data within the department is accessible, reliable and secure, enabling data-driven decision making. As an SEO Data Engineer, you will work with autonomy to translate user requirements and operational needs into production-ready data architectures while supporting best practices across the team.

As a Data Engineer, you will:

• Write clean, efficient, reusable and tested code, applying best practices, version control and CI/CD automation.
• Design, implement and maintain robust ETL/ELT pipelines and data flows connecting operational systems to analytics and Business Intelligence (BI) platforms using Databricks.
• Design and maintain structured data models to ensure data is conformed, efficient to query and business-ready.
• Monitor, troubleshoot and optimise pipeline performance and system reliability, investigating root causes and putting preventative measures in place.
• Collaborate with data analysts, data scientists and business stakeholders to gather requirements and deliver business value through iterative agile delivery.
• Apply data classification, security standards, and governance procedures to manage access controls, maintain metadata and ensure regulatory compliance.

We are committed to supporting professional development and will provide opportunities for learning, mentoring and skills development throughout the appointment.