Senior Data Engineer

Lead Data Engineer – Up to £110,000 – Fully Remote (UK)

My client, an AI‑driven eLearning platform, used globally by organisations seeking scalable, intelligent, and adaptive training solutions. Their technology leverages automation, advanced analytics, and AI‑powered personalisation to transform how enterprises deliver learning  the company continues to expand and strengthen its data capabilities, they are now searching for a Lead Data Engineer to drive the next phase of their platform evolution.

As the Lead Data Engineer, you will own the architecture, optimisation, and evolution of my client’s cloud‑native data ecosystem. This is a hands‑on leadership role where you'll guide engineering direction, ensure enterprise‑grade security and compliance, and shape the technical foundation that supports a rapidly scaling AI learning platform.

You will operate as the most senior engineer within the data function, partnering with product, engineering, security, and client‑facing teams to build a robust, modern, and scalable data platform.

• Salary:
  Up to £110,000
• Working arrangement:
  Fully Remote (UK)
• Competitive annual leave package
• Company pension scheme

• Lead the modernisation of the data platform, including enhacing the utilisation of Databricks, improving platform scalability and processing reliability.
• Design and implement AWS-native ETL/ELT pipelines using S3, Glue, Athena, and Step Functions to support reliable data flows across global enterprise customers.
• Build and maintain Lakehouse architecture using Apache Iceberg to support scalable analytics with ACID compliance and schema evolution.

• Implement enterprise-level data validation workflows that reduce downstream data issues.
• Drive platform-wide security compliance, including encryption at rest/in transit, secure deletion automation, and infrastructure security baselines.

• Reduce platform costs through workload optimisation and access pattern redesign.
• Improve platform operational efficiency and reduce cloud costs via architectural streamlining and modernisation.

• Mentor engineers, contribute to hiring, and help scale the team to meet growing product demands.

You will excel in this role if you bring:

• Expertise in Databricks, Spark, AWS Glue, Athena, and Iceberg/Lakehouse architectures.
• Strong understanding of data security, compliance, and enterprise governance.
• A track record of cost optimisation, performance tuning, and scalable design.
• Experience leading engineering initiatives or mentoring teams.
• Hold active SC Clearance, or have eligibility for SC Clearance.