Lead Data Engineer - GCP Platform Services & Frameworks

About this role:

The COO Technology group delivers technology solutions for the Chief Operating Office, supporting operations, control executives, strategic execution, business continuity, resiliency, data services, regulatory relations, customer experience, enterprise shared services, supply chain management, and corporate properties. Our mission is to modernize and optimize technology platforms for these critical functions.

Within COO Technology we are seeking a Lead Data Engineer who will be responsible for designing GCP‑based data pipeline development, semantic layers, and reporting capabilities that support front‑office, middle‑office, and finance functions. The role will ensure data is accurate, reconciled, well‑governed, and consumable via standardized and self‑service analytics and dashboards, as well as consumable APIs for downstream systems.

In this role, you will:

* Lead the design and implementation of scalable, secure data platforms on Google Cloud using managed services (Big Query, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Composer)

* Build reusable frameworks and tooling (ingestion, transformation, quality, orchestration) that can be adopted by multiple product and domain teams

* Develop shared transformation libraries in Python/SQL/Beam (e.g., common SCD patterns, data quality checks, masking/tokenization routines)

* Provide orchestration capabilities via Cloud Composer or Cloud Workflows with reusable DAGs/templates and CI/CD integration

* Optimize cost, performance, and reliability of GCP data workloads (partitioning, clustering, storage classes, autoscaling strategies)

* Build opinionated data ingestion frameworks (e.g., config‑driven pipelines, connectors, schema handling, error handling) on top of Dataflow, Dataproc, or Composer

* Implement robust data modeling (dimensional, data vault, or canonical models) and semantic layers in Big Query and related tools

* Enforce data quality, lineage, and observability using standardized metrics, validation rules, and monitoring dashboards

* Partner with domain data engineers, analytics, and ML teams to onboard use cases onto platform services and frameworks

* Document patterns, runbooks, and best practices, and provide enablement through workshops and code examples

* Contribute to platform roadmap, tool selection, and evaluation of new GCP services and open‑source components

Required Qualifications:

* 5+ years of Data Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education

* 4+ years with orchestration tools (Cloud Composer/Airflow) and CI/CD (Cloud Build, Git‑based workflows) for data workloads

* 4+ Python and Spark for building data pipelines, libraries, and automation tooling

* 4+ years of advanced SQL and data modeling for analytics and warehousing; experience with ETL/ELT design patterns

* 4+ years' experience with automated testing, data quality checks, and monitoring for pipelines and platform services

* 4+ years' experience with logging/monitoring stacks (Cloud Logging, Cloud Monitoring, error reporting, metrics dashboards

* 3+ years of experience with core GCP data services:
Big Query, Dataflow/Apache Beam, Dataproc, Pub/Sub

Desired

Qualifications:

* Bachelor's degree in computer science, Engineering, or related field, or equivalent

* Experience with knowledge graphs and semantic search

* Excellent communication and presentation skills across technical audiences

* Advanced Data Modeling skills, including data modeling in tabular and MPP models, design, composite models, and performance optimization of large, complex datasets

* Strong SQL expertise and hands‑on experience with Starburst and/or Trino, including connector configuration, query optimization, and secure data federation across GCP data stores

* Collaborative mindset with a focus on partnering across business, finance, risk, and technology teams to deliver high‑quality, reconciled data, APIs, and reporting outcomes

* Experience in large financial services organizations (100k+ employees)

* Experience with Agile transformations and technology roadmaps

* Experience working with onshore and…