Principal Data Engineer

Job Title

Principal Data Engineer

IG Group is a FTSE 100 fintech operating across five continents, serving over 1.3m customers and handling billions of dollars in transactions – built on scale, trust, and proof. We didn’t pivot to innovation; it’s how we’ve always operated. What that means for the people who work here is real: genuinely complex problems to solve, the technology and resources to tackle them properly, and the kind of scope that’s rare in established businesses.

The bar is high – bring a curious and forward-thinking mindset and we’ll give you the platform to define what comes next. Join us at IG – the future gets built here.

IG’s Data function is a central capability serving both central and divisional business lines across the entire IG estate. The Data Engineering team is responsible for the platforms that underpin analytics, reporting, compliance, and client-facing data products, and that will increasingly power AI and machine-learning use cases across the firm.

The current data landscape reflects IG’s growth: a combination of GCP-native capabilities, on-premises systems, and a legacy AWS data platform. The strategic direction — endorsed by the CDO and Executive — is clear: consolidate onto GCP, raise the bar on data quality, and enable self-service access to trusted data for teams across the business.

• Own the technical architecture of IG’s GCP data platform end-to-end — Core Lake (Big Query), batch and real-time (Kafka) ingestion, transformation in the Medallion architecture, and Data-Engineering-owned integration services — setting the reference designs and patterns squads build against.
• Lead delivery of the most complex, high-risk work within the GCP consolidation programme, including migration off the legacy AWS platform and on-premises systems onto GCP-native capabilities.
• Define the platform contracts, interfaces, and self-service patterns that let mature divisions build and own their own models safely against trusted Core Lake data, and ensure fully-managed builds for less-mature functions follow the identical pattern so they are handover-ready by design.
• Drive data quality as a first-class, firm-wide concern: establish data contracts, observability, SLA/SLO monitoring, and automated alerting and remediation across ingestion and transformation layers, and hold squads to those standards.
• Act as the senior technical point of contact for stakeholders across Compliance, Marketing, Finance, Data Science, and the self-serving divisions (UK, APAC, tastytrade/US, Quants, Risk), translating requirements into well-scoped, well-architected engineering deliverables.
• Set and uphold engineering standards through code and design reviews, technical mentoring, and reusable frameworks that raise the bar across squads in London, India, and Poland — growing the technical capability of the whole function.
• Provide technical direction and quality assurance for third-party staff augmentation, ensuring augmented capacity engaged for time-boxed projects works to IG’s patterns and leaves behind maintainable, standards-compliant assets.
• Maintain delivery rigour: shape sprint cadences, elevate technical risk early, and report on pipeline SLOs, data freshness, and DORA-style engineering metrics to the Head of Data Engineering and CDO.

• Hands-on experience designing and operating large-scale data pipelines on GCP (Big Query, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures
• Demonstrated experience leading or technically mentoring a team of data engineers, with the ability to set standards, conduct code and design reviews, and grow engineers’ capabilities
• Strong grasp of data quality practices: data contracts, pipeline observability, SLA/SLO definition, and automated alerting and remediation
• Solid understanding of SQL transformation patterns and modern tooling such as dbt, alongside experience managing ingestion estates with third-party connectors and in-house integrations
• Clear and confident communicator able to work cross-functionally with non-technical stakeholders;…