Senior Data Engineer

Senior Data Engineer

Salary - $125,000 (Chicago)

At Retail Insight, we're transforming the way some of the world's biggest retailers and CPG brands use data, analytics and technology to make smarter decisions.

Our people are at the heart of what we do. We bring together talented people with different experiences, perspectives and expertise to solve complex problems, challenge the status quo and make a real impact for our clients.

We're growing, and we're looking for a Senior Data Engineer to join our Engineering team.

We're looking for an experienced Senior Data Engineer to build and support the reliable, scalable data systems that power our solutions for US retail clients.

As our US client portfolio grows, you'll work with complex data environments, helping translate business requirements into practical technical solutions. Working closely with our Chicago-based Lead Data Engineer, alongside data scientists, analysts and product teams, you'll build data pipelines and products that help our clients make better decisions.

This is a hands-on role combining strong technical skills with collaboration and clear communication. You'll contribute to the evolution of our data platform, improve reliability and performance, and help maintain high engineering standards.

You’ll also work with colleagues and client stakeholders to understand their needs and support successful delivery.

• Building and maintaining scalable data pipelines, developing and optimising batch and streaming workflows for reliability and performance.
• Contributing to the design and implementation of data architecture.
• Implementing data quality checks, monitoring and observability across pipelines.
• Applying security, access control and governance standards established by our platform team.
• Collaborating with data scientists and analysts to deliver data products that support their work.
• Working with the Lead Data Engineer to translate business requirements into technical solutions.
• Participating in code reviews, setting a high bar for engineering quality and contributing to team standards.
• Helping identify opportunities to optimise cloud costs and platform performance.
• Contributing to platform documentation, tooling choices and continuous improvement.

• Solid, demonstrable experience in data engineering, including building and maintaining production-grade Databricks pipelines.
• Experience with Databricks Unity Catalog and Databricks Asset Bundles.
• Strong hands-on knowledge of Apache Spark, Delta Lake and Databricks Workflows.
• Proficiency in Python and SQL, with a good understanding of data modelling, ETL/ELT patterns and distributed data processing.
• Experience with Azure data platforms, including Azure Data Factory, ADLS Gen2 and Azure Dev Ops, or equivalent services on AWS or GCP.
• Experience implementing data quality checks and pipeline monitoring.
• Strong communication skills and the ability to work effectively with technical and non-technical colleagues.
• A collaborative approach, ownership of quality outcomes and a drive to learn and improve.
• A degree in computer science, data engineering or a related field, or equivalent practical experience.
• Willingness to travel as required.

• Familiarity with infrastructure-as-code tools such as Terraform or Bicep.
• Familiarity with dbt or similar data transformation frameworks.
• Exposure to real-time streaming technologies such as Kafka or Event Hubs.
• Exposure to retail or consumer packaged goods (CPG) data, such as point of sale, inventory or supply chain.
• Experience working directly with clients.

We believe in flexibility, growth, and taking care of our people, both in and out of the office.

Hybrid…