Databricks Data Engineer

EXL (NASDAQ: EXLS) is a global data and AI company that offers services and solutions to reinvent client business models, drive better outcomes and unlock growth with speed. Bringing together domain expertise with robust data, powerful analytics, cloud and AI to create agile, scalable solutions and execute complex operations for the world’s leading corporations.

EXL was founded on the core values of innovation, collaboration, excellence, integrity and respect, creating value from data to ensure faster decision-making and transforming operating models. Key industries include Insurance, Healthcare, Banking and Financial Services, Media, and Retail, among others.

Headquartered in New York, our team is over 60,000 strong, with more than 50 offices spanning six continents. For information, visit

Location: Leeds, United Kingdom (Hybrid, 1-2 days in the office)

Employment Type:Permanent to start ASAP

Summary of the role:

We’re looking for a highly skilled Azure Databricks Data Engineer to design, develop and deliver cloud-based data integration and analytics solutions within our Retail portfolio
.

You’ll work across data engineering, modelling and transformation initiatives, building scalable pipelines, optimising performance and ensuring best practices in data quality and governance. This is a hands-on technical role requiring strong Azure and Databricks experience, coupled with a solid understanding of Retail data and business processes
.

As part of this role, you’ll have the opportunity to work on high-impact Retail data transformation programmes
, collaborating with experienced data professionals in a forward-thinking, innovative environment. You’ll gain exposure to cutting-edge Azure and Databricks technologies
.

• Design, build and deploy scalable data pipelines using Azure Databricks
  , Azure Data Factory and Azure SQL Database
  .
• Optimise Databricks environments for performance, cost and security.
• Develop and maintain data integration
  , ETL/ELT processes and data quality frameworks.
• Translate retail business requirements into data models, ensuring consistency and accuracy across data layers.
• Partner with business and technology stakeholders to deliver effective data solutions, analytics and insights that support retail operations and decision-making.
• Create technical documentation including data flow diagrams, architecture designs and test plans.
• Support the delivery of Retail data transformation initiatives, ensuring adherence to data governance and best practice standards.
• Stay current with emerging data engineering and Azure ecosystem technologies to drive continuous improvement.

Retail industry experience is essential, with a strong understanding of retail data domains, including customer, product, sales, inventory and supply chain data.

Minimum of 5 years' management experience required, leading technical delivery or overseeing project teams within data engineering or integration environments.

8-10 years' experience in data engineering and integration, ideally in enterprise-scale environments.

Hands-on experience with Spark (PySpark/SparkSQL), ETL/ELT design and data pipeline orchestration.

Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts.

Experience working with large, complex retail datasets and distributed computing environments.

Proficiency in SQL, version control (e.g. Git) and CI/CD pipelines.

Familiarity with Agile delivery and Dev Ops practices (Azure Dev Ops, JIRA, etc.).

Preferred

Skills & Experience:

• Knowledge of EDM (Enterprise Data Management) practices, data governance, metadata management and lineage tools.
• Exposure to…