Lead Data Migration Engineer

The team you'll be working with:
We are seeking an accomplished and detail-oriented Lead Data Migration Engineer to join our Data & AI practice. The successful candidate will bring deep expertise in data migration, ETL/ELT pipeline engineering, and cloud-native data platforms, with a strong focus on AWS-based Data Lakehouse architectures.

This role is critical in designing, building, and validating end-to-end data migration pipelines, enabling the seamless transition of data from legacy data warehouses to modern cloud platforms. You will play a key role in ensuring data quality, integrity, and performance, particularly through robust testing and validation of ETL pipelines.

As a senior practitioner, you will collaborate with architects, engineers, and analysts to deliver secure, scalable, and high-performing migration solutions, leveraging technologies such as AWS Glue, Apache Iceberg, Python/PySpark, SQL, and YAML-based configurations.

You will thrive in a collaborative, client-facing environment, with a passion for solving complex data challenges, ensuring delivery excellence, and driving modernisation through data migration initiatives.

What you'll be doing:
Client Engagement & Delivery

Work as a senior engineer within data migration programmes, supporting delivery across complex transformation initiatives

Collaborate with architects and stakeholders to implement migration strategies and technical solutions

Contribute to project planning, estimation, and execution of migration deliverables

Data Migration Engineering

Design, build, and optimise data migration pipelines from legacy data warehouses to AWS-based Data Lakehouse platforms

Develop and maintain ETL/ELT pipelines using:
AWS Glue Python / PySparkSQLYAML-driven configurations

Implement bulk data migration processes followed by incremental/delta loads

Support pipeline repointing and replatforming to target cloud architectures

Data Pipeline Testing & Validation (Core Focus)

Test end-to-end ETL data solutions running on AWS services, including AWS Glue and Apache Iceberg Validate data feed pipeline migrations transitioning to new AWS Data Lakehouse architectures

Test pipelines underpinned by:
AI ETL accelerators

Python/PySpark transformations SQL-based logicYAML configuration-driven orchestration

Execute and validate:
Initial bulk data loads

Subsequent incremental/delta migrations

Develop and run SQL-based validation and reconciliation queries to ensure:
Data completeness

Data accuracy

Transformation correctness

Support creation of automated testing and validation frameworks

AWS Data Platforms & Lakehouse

Work hands-on with AWS cloud services, including:
AWS GlueS3-based data lakes

Support implementation of Data Lakehouse architectures, including Apache Iceberg Optimise data pipelines for performance, scalability, and reliability in cloud environments

Data Modelling & Transformation

Apply data transformation logic aligned to migration mapping rules

Support data modelling activities required for target-state schemas

Ensure consistency between source and target systems during migration

Collaboration & Best Practices

Collaborate with:
Solution Architects Data Engineers Data Migration Architects Analysts and QA teams

Promote engineering best practices, reusable components, and automation

Contribute to migration accelerators and reusable frameworks

Quality, Governance & Security

Ensure data integrity, quality, and traceability throughout migration processes

Follow best practices for secure data handling in regulated environments

Support compliance with:
GDPRUK public sector data requirements (where applicable)
Contribute to audit and validation processes

What experience you'll bring:
Proven experience in data engineering and data migration delivery, particularly within cloud environments

Strong focus on data pipeline testing, validation, and quality assurance

Ability to work across the full data lifecycle, with emphasis on migration and transformation

Strong analytical, problem-solving, and communication skills

Experience working in client-facing and delivery-focused environments

Ability to mentor junior engineers and contribute to team delivery

Technical Expertise

Strong hands-on experience with:
AWS cloud services, especially AWS Glue Python / PySpark for data transformation SQL for querying, validation, and reconciliationYAML configuration for pipeline orchestration

Experience testing and validating:
End-to-end ETL/ELT pipelines

Data migration workflows

Familiarity with:
Apache Iceberg and Lakehouse architectures

Distributed processing frameworks (e.g., Apache Spark)
Data lake environments

Understanding of:
ETL vs ELT design patterns

Migration performance tuning

Experience with version control and CI/CD tools desirable

Who we are:

We’re a business with a global reach that empowers local teams, and we undertake hugely exciting work that is genuinely changing the world. Our advanced portfolio of consulting, applications, business process, cloud, and infrastructure services will allow you to achieve great…