Data Engineer

The client is looking to appoint an experienced Data Analyst to join the Data and Commissioning Team within the Employment and Skills Directorate.

• Support the engineering, optimisation, and sustainability of the Authority’s Databricks-based Lakehouse platform.
• Develop high‑quality ETL pipelines and transformation processes to ensure accuracy, efficiency, and compliance.
• Reverse‑engineer and rationalise existing Python-based pipelines developed by external partners.
• Document data flows, dependencies, and processes to enable effective BAU handover and maintainability.
• Provide technical assurance, mentoring, and clear documentation to build long‑term internal capability.
• Implement best‑practice data models (e.g., medallion architecture, dimensional modelling).
• Ensure data structures support self‑service analytics, BI reporting, and advanced analytics.
• Contribute to evolving architectural standards and enhance the enterprise data model.

• Strong ETL design, optimisation, and debugging experience.
• Advanced Python for ETL development and automation.
• Strong SQL for validation, transformation, and optimisation.
• Experience with Azure Data Factory for ingestion and scheduling.
• Proficiency with Git Hub and collaborative development workflows.

If you are available and interested, please apply today!