Data Platform Lead: AWS, Databricks & Pipelines

Join us as an Engineering Lead – Enterprise Data Platform for Barclays Customer, Comms and Mar Tech team, where you will play a pivotal role in leading the technical design and delivery of data solutions supporting the migration to a cloud-based Enterprise Data Platform (EDP).

You will be responsible for shaping how Customer, Comms and Mar Tech data is provisioned and configured in the Data Lab environments, integrated and consumed, enabling scalable, secure, and efficient data capabilities across the organisation.

This role sits at the heart of a strategic transformation to replace legacy data warehouses and platforms with a modern cloud-native EDP. You will lead engineering solution design, define new data architecture patterns including orchestration, and scheduling, and ensure seamless integration with downstream systems while maintaining high standards of data quality, security, and operational resilience.

The role will include; leading the design and build of scalable data pipelines and integrations on AWS, driving the migration of customer and Mar Tech datasets from legacy platforms to the EDP, and establishing robust Dev Ops and Data Ops practices to ensure production-ready, reliable, and high-performing data products.

To be successful as an EDP Customer Engineering Lead, you should have:

• Strong hands‑on expertise in Python, PySpark, SQL, and Databricks, with the ability to build and optimise scalable data pipelines and analytics‑ready datasets
• Experience implementing CI/CD and Dev Ops practices for data platforms, including Git Lab workflows, automated testing, deployment pipelines, and environment management
• Proven experience delivering AWS solutions including cloud infrastructure with a focus on performance tuning, monitoring, data quality, security, and cost optimisation

Some other highly valued skills may include:

• Understanding of Enterprise Data Platform concepts, including data products, domains, and consumption patterns with a focus on operability and production readiness
• Awareness of data security practices, particularly in handling sensitive data using governed and automated approaches
• Experience with file‑based integration patterns, including managing constraints around file formats, splitting, and downstream system limitations

You may be assessed on the key critical skills relevant for success in role, such as risk and controls, change and transformation, business acumen strategic thinking and digital and technology, as well as job‑specific technical skills.

This role is based in Knutsford.