Data Engineer IR35

Position: Staff Data Engineer |  | Contract | Inside IR35
Staff Data Engineer | AWS | PySpark | Data Modernisation | London | Contract | Inside IR35

We're supporting a global, technology-led organisation undertaking a large-scale cloud data transformation programme.

This isn't a traditional Data Engineering role. The organisation is modernising hundreds of complex data pipelines, establishing reusable engineering patterns and embedding engineering excellence across its Data function.

As a Staff-level engineer, you'll combine deep hands-on technical expertise with organisation-wide technical influence, helping shape how modern Data Engineering is delivered across multiple engineering teams.

This is an opportunity to work on enterprise-scale data movement, cloud-native architectures and AI-assisted software engineering within a highly collaborative engineering culture.

You'll be responsible for
Defining technical standards and engineering patterns for a large-scale Data Modernisation programme
Re-engineering complex legacy data pipelines into production-grade Py Spark
Building scalable, reusable data movement frameworks across multiple engineering teams
Driving best practice around AWS Glue, Apache Spark and Apache Airflow
Designing high-quality testing frameworks covering performance, data quality and reliability
Leveraging Agentic AI development tools to improve engineering productivity while maintaining code quality
Coaching and influencing experienced engineers through technical leadership rather than line management
Working closely with enterprise architecture and platform teams to improve how data is governed, managed and consumed

We're looking for someone with experience in
Staff, Principal or Lead Data Engineering roles
Python and expert-level Py Spark
AWS Glue
Apache Spark
Apache Airflow
Cloud-native Data Engineering architectures
Large-scale data pipeline modernisation
Building reusable engineering patterns within enterprise environments
Data testing, validation and observability
AI-assisted software development using modern coding assistants
Working within regulated or highly governed organisations

More important than any individual technology is your ability to combine hands-on engineering excellence with technical leadership, helping multiple engineering teams adopt consistent, scalable engineering practices.
Contract
£(Apply online only) per day (inclusive)
Initial 6-month contract with significant programme runway
Inside IR35
London (3 days per week on-site)