Lead Data Engineering; Hands-On Python​/Databricks

Job Title:

Technical Delivery Lead
- Data Engineering Lead (Hands-on Python / Databricks)

Contract:

6 months
- Potentially extension.

Location:

London Hybrid (5 days in 10 in the office)

Rate:
Highly competitive

Status:
Inside IR35

Working pattern:
Full time

• Proven experience in Data Engineering or Software Engineering.
• Previous experience in a Technical Lead, Lead Data Engineer, or Technical Delivery Lead position.
• Strong hands‑on expertise with Python.
• Extensive experience with Databricks, including Workflows, Notebooks, and Delta Lake.
• Strong experience with Apache Spark /PySpark.
• Proven track record building and optimising large‑scale ETL/ELT pipelines.
• Strong understanding of modern data architecture and Lakehouse principles.
• Excellent SQL skills and experience working with distributed data systems.
• Experience working with cloud platforms such as AWS, Azure, or GCP.
• Strong Agile delivery experience.
• Experience leading technical teams and delivering complex projects.
• Expected split: ~30-50% hands‑on coding + technical problem‑solving.

• Lead end‑to‑end delivery of data engineering solutions.
• Design and build scalable data pipelines using Databricks and Azure.
• Develop and optimise solutions using Python and PySpark.
• Drive engineering best practices, code reviews, and technical standards.
• Work closely with architecture, business, and technology stakeholders.
• Mentor and support engineers within the team.
• Contribute to data platform and Lakehouse architecture decisions.

• Approximately 60% hands‑on engineering and 40% technical leadership.
• Proven experience leading data engineering teams.
• Strong stakeholder management and communication skills.
• Experience delivering greenfield data platform initiatives.

• Work on innovative projects that make a difference in the financial sector.
• Collaborate with a talented team in a vibrant environment.
• Enjoy a competitive daily rate and flexible working arrangements.

We respect and appreciate people of all ethnicities, generations, religious beliefs, sexual orientations, gender identities, and more. If you require reasonable adjustments at any stage, please let us know and we will be happy to support you.