Data Engineering Lead

Salary: £9,750 - 10,400 per year

• Strong expertise in Azure data services, including ADLS, ADF, Databricks, Synapse, Event Hub, and Kafka.
• Hands-on experience with Python, SQL, and Spark/PySpark.
• Experience with data modeling, metadata, lineage tooling, performance and cost optimization.
• Proven leadership experience managing engineering teams.
• Experience working in Agile and Dev Ops environments, including CI/CD and test automation.
• Exposure to energy trading systems such as ETRM or Endur, market data sources such as ICE, Platts, Bloomberg, and SAP.
• A degree in Computer Science, Engineering, or equivalent practical experience.
• Azure Data Engineer or Databricks certifications are desirable.

• Lead the data acquisition function by defining ingestion strategy and scalable batch and streaming data patterns.
• Design and deliver Azure-based data pipelines and platform solutions.
• Standardise connectors, schemas, controls, CI/CD, infrastructure as code, and automated testing.
• Lead onboarding of enterprise and market data sources, ensuring service levels and observability are in place.
• Partner with Product Owners, Architects, and business stakeholders on data design and governance.
• Act as the domain Data Manager for lineage, metadata, quality, access and master data ownership.
• Ensure compliance with data lifecycle, audit, records management and governance standards.
• Drive engineering best practices, operational excellence and continuous improvement.
• Manage capacity planning, skills development and vendor contributions.
• Translate end-user requirements into scalable engineering solutions for Trading, Risk, Finance and Operations.

• Azure
• CI/CD
• Databricks
• Dev Ops
• Kafka
• Marketing
• Python
• Py Spark
• SAP
• SQL
• Spark
• Cloud

We are a hiring company based in London offering an initial contract through to the end of the year, with onsite working required three days per week. We are looking for a Data Engineering Lead to help shape and deliver reliable, compliant, and high-value data solutions across Trading, Risk, Finance, and Operations. In this role, you will join a team focused on data governance, engineering excellence, and scalable Azure-based solutions.

We also work with businesses across the UK in Finance, Marketing, IT, and Engineering, and we are committed to helping people find roles where they can make an impact and grow their careers.