Senior Data Engineer

We are a team of tech enthusiasts on a mission to bring together the best minds in IT services and analytics. Our goal? To build cutting-edge IT and Analytical Hubs that empower our partners to become truly data-driven organizations.

For our client’s Data Management & Data Foundation area, we are looking for an exceptional Senior Data Engineer – a highly experienced, hands‑on engineer who can independently take ownership of complex technical challenges and turn them into robust, scalable solutions.

This is not a maintenance‑focused or business‑as‑usual role. You will join a high‑seniority engineering group working on strategic and technically challenging initiatives, moving between projects depending on priorities and where the strongest engineering expertise is needed.

The wider Data Management organization consists of approximately 30 engineers and technology specialists, while this role will focus primarily on a selected set of high‑impact initiatives.

• Design, develop, and optimise complex data solutions and data pipelines on Azure and Databricks.
• Take end‑to‑end technical ownership of assigned engineering problems – from understanding the challenge through solution design, implementation, testing, and production deployment.
• Write high‑quality, production‑grade code using Python, PySpark, and SQL.
• Work extensively with Azure Databricks and modern capabilities of the Databricks ecosystem.
• Explore and implement AI‑enabled capabilities within the data platform, including solutions involving Databricks AI features, RAG‑based approaches, data discovery, and conversational access to enterprise data.
• Contribute to the migration and modernisation of existing data platforms and solutions into Databricks‑based architectures.
• Work with Unity Catalog and contribute to scalable approaches to data discovery, governance, and access.
• Solve technically difficult or non‑standard problems where there may not be an existing blueprint or ready‑made solution.
• Proactively identify opportunities to improve architecture, performance, scalability, reliability, and maintainability.
• Challenge existing approaches when you see a better technical solution and actively contribute your own ideas.
• Collaborate with Data Engineers, Architects, Product Owners, Data Ops, QA, Dev Ops, and other technology teams.
• Support architectural discussions and contribute to technical design decisions across the Data Management landscape.
• Move between strategic initiatives as priorities evolve, bringing strong engineering expertise wherever it is most needed.

This is a role for someone who enjoys building things, solving difficult engineering problems, and having a real technical impact rather than simply executing predefined tasks.

• 8+ years of professional experience in Data Engineering, Software Engineering, or a closely related field.
• Strong, hands‑on commercial experience with Azure Databricks.
• Very good knowledge of the Azure cloud ecosystem and cloud‑based data architectures.
• Excellent programming skills in Python – we are looking for a strong coder, not someone who only uses Python for basic notebook development.
• Advanced PySpark skills and experience working with large‑scale data processing workloads.
• Advanced SQL, including complex transformations, optimisation, and analytical/window functions.
• Strong understanding of data engineering principles, ETL/ELT patterns, data pipelines, distributed processing, and production data platforms.
• Experience designing and delivering solutions independently, with limited supervision.
• Ability to understand an unfamiliar technical problem, propose an approach, and drive it through to implementation.
• Strong troubleshooting and problem‑solving skills.
• Experience…