Principal Data Engineer; Databricks

Role Overview

Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands‑on role, combining the building and optimisation of production data pipelines with technical leadership needed to raise engineering standards and develop the people around them. The role functions as a key interface between a) business stakeholders and department heads, b) the wider data and analytics team, and c) owners of source systems and data in Finance and IT.

• Designing and building robust, production‑grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP)
• Working "under the hood" to optimise the platform for performance and cost – tuning compute, jobs and queries, managing storage and table layout, and keeping platform spend under control
• Defining and maintaining consistent business metrics and a semantic layer (for example, using Metric Views) so reporting is built on trusted, reusable definitions
• Integrating data from core business systems, including SAP S/4
  
  HANA, and other sources, using tools such as Fivetran (both SaaS connectors and HVR)
• Setting and upholding engineering standards across the team – code quality, testing, documentation, CI/CD and data governance
• Coaching and mentoring junior and mid‑level engineers, reviewing their work and helping them develop
• Partnering with business stakeholders to understand their needs, shape practical solutions and ensure the platform delivers genuine value
• Engaging data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiating data quality improvements where required

Skills & Experience

• At least 3 years of hands‑on, daily Databricks experience, covering both pipeline development and under‑the‑hood performance and cost optimisation
• Up to date with recent platform developments, such as Spark Declarative Pipelines (SDP) and Metric Views
• Strong proficiency in programming languages commonly used in data engineering, such as Python, SQL and Spark
• Advanced experience with data manipulation, data modelling, database design and query optimisation
• Experience managing or coaching junior developers
• A track record of managing and influencing business stakeholders
• Experience working with SAP S/4
  
  HANA data sets, Fivetran (including both SaaS connectors and HVR), Power BI semantic modelling would be beneficial

• Combining deep, hands‑on engineering skill with sound judgement about cost, performance and long‑term maintainability
• Coaching, mentoring and raising the capability of less experienced engineers
• Collaborating, communicating confidently and influencing business stakeholders
• Breaking down complex technical concepts and explaining them simply to non‑technical audiences
• Staying current with a fast‑moving platform and bringing new capabilities into everyday practice
• Taking ownership and driving work independently, from concept through to production

This role requires full‑time office‑based attendance, five days per week.

Qualified candidates to apply by uploading a cover letter and a recent resume by close on business 31 July 2026.