Lead Data Engineer; Databricks

Lead Data Engineer - Databricks
Up to £75,000 | Permanent | Huddersfield | 4 days onsite
Want to build the data platform rather than just maintain one?

This is an opportunity for a hands‑on Lead Data Engineer to play a key role in a major data transformation - moving a complex, multi-site UK organisation away from fragmented Legacy data and manual processes towards a modern Databricks environment
.There's plenty to build, plenty to improve and a genuine opportunity to influence how Data Engineering is done.

We're looking for someone who loves Data Engineering, can lead from the front technically and wants to take ownership of a small engineering team while remaining close to the code.

The role
As Lead Data Engineer
, you'll take day-to-day ownership of engineering delivery across the Databricks platform, working closely with the Head of Data Engineering and leading a small team of Data Engineers. You'll be joining at an important point in the organisation's transformation. Databricks is becoming the strategic data platform, with a significant programme underway to create better engineered pipelines, trusted data models and reusable data products.

This is not a hands-off management role
. You'll remain technically involved - designing solutions, building pipelines and solving engineering problems - while providing the direction, standards and leadership the wider team needs.

Over time, you'll become a key internal technical authority for Databricks, working closely with Data Architecture and senior data leadership to shape how the platform evolves and what gets built next.

What you'll be doing

• Leading Data Engineering delivery across Databricks
• Remaining hands‑on with engineering, development and problem-solving
• Designing and building scalable, reliable data pipelines
• Helping establish effective bronze, silver and gold data layers
• Developing curated, trusted and reusable datasets
• Improving data quality, structure and provenance
• Supporting the development of enterprise data models and single sources of truth
• Leading, mentoring and developing a small team of Data Engineers
• Prioritising and allocating engineering work across multiple projects
• Establishing strong engineering standards and ways of working
• Working closely with Data Architecture, Analytics and wider technology teams
• Acting as a technical sounding board to the Head of Data Engineering
• Helping reduce manual data processes through better engineering and automation
• Building internal Databricks knowledge and capability
• Deputising for senior Data Engineering leadership when required

The environment
This is a large, complex and highly distributed organisation with a substantial data estate spanning multiple business areas, locations, systems and data sources. There is a significant roadmap of transformation ahead, with opportunities to improve how operational, customer, commercial and product data is brought together and used across the organisation. A major focus is reducing fragmented and manual ways of working and creating properly engineered, trusted data that can support everything from day-to-day decision-making through to advanced analytics, automation and future AI initiatives.

You'll be joining while that environment is being built - not once all the interesting decisions have already been made.

What we're looking for

We're looking for a Data Engineer first and a leader second - someone who has progressed into technical leadership but still genuinely enjoys building.

You'll ideally bring:

• Strong commercial Data Engineering experience
• Hands‑on Databricks expertise
• Excellent SQL skills
• Experience designing and developing production data pipelines
• Strong understanding of modern data-platform architecture
• Experience working within complex enterprise data…