SC Cleared Data Engineer

Job Title:

Data Engineer - Active SC Clearance Required

Location:

Leeds, 1x per week Duration: 3 months with strong chance of extension

Contract:

Outside IR35 Current active SC Clearance is essential. Applications from candidates who are only eligible for SC Clearance cannot be considered. This is a great opportunity for an experienced Data Engineer to support the delivery of a greenfield Azure cloud data platform for a major transformation programme. This contract will see you building a modern Azure data platform that brings together complex data from multiple systems, improving data quality, automation and reporting while helping create a single, trusted view of customer information.

Key Responsibilities Design, build and optimise Azure data pipelines using Azure Data Factory and Azure Databricks Develop data ingestion solutions using Change Data Capture and Managed File Transfer Build canonical data models, conformed dimensions and fact tables Deliver curated Gold-layer datasets for reporting, analytics and AI use cases Automate data validation and quality assurance processes, replacing manual checks Configure Unity Catalog, clusters, pipelines and data contracts Contribute to data governance, security, lineage and platform design Collaborate on solution design, code reviews, testing and defect resolution Essential Skills & Experience Current active SC Clearance (mandatory) Strong commercial experience with Azure Databricks, Azure Data Factory and Azure data engineering Advanced Python (including Pandas) and SQL Hands-on experience with Unity Catalog Experience designing data warehouses, including slowly changing dimensions (SCD) and dimensional modelling Knowledge of CDC, data ingestion patterns and governed data platforms Understanding of data governance, security and lineage

Experience with in regulated or public sector environments is advantageous Familiarity with Power BI is beneficial but not essential