Data Engineer

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.##

** Hybrid

Location:

New York**##
** Job Description
** Key Responsibilities  Ingestion & Bronze  Build and operate ingestion into a Delta raw landing zone on GCS using Lakeflow Connect managed connectors, Zerobus, structured streaming, and batch ETL/ELT  Ingest transaction signals, behavioural events, and third-party identity graphs
- Live Ramp RampID, UID2, GCLID chains, and household device graphs  Own schema enforcement and evolution at the landing boundary, so upstream change surfaces as a controlled event rather than a downstream failure

Unstructured Processing (Bronze → Silver)  Build the SLM/LLM document-processing tier that converts unstructured blobs
- Adobe AEM exports, asset libraries, PDFs, brand guidelines, compliance rule sets, creative - into governed Silver tables  Implement parsing and OCR, entity extraction, data hygiene and deduplication, normalisation and mapping, schema inference and enforcement, and PII detection and masking as pipeline stages with measurable quality gates  Treat model-based extraction as a transformation step under test, with validation and deterministic fallbacks - not as a black box Silver
- Identity, Features, Model Serving  Implement identity resolution at scale: deterministic matching, probabilistic graph construction, and household- and device-level cluster assembly across 1B+ data points  Own feature engineering and the data-quality expectations framework - reconciliation, lineage, deduplication - reducing manual reconciliation cycles from weeks to hours  Build and operate the serving layer behind the analytical model agents: MTA (7- and 28-day), MMM, incrementality, LTV:

CAC, cohorts, reach/frequency, cobrand overlap, channel viability, card activation, product holdings, scenario reallocation, and channel universe Gold
- Serving & Clean Room  Maintain clean room connectors and privacy-preserving data exchange pipelines across GMP, AMC, XMi, Live Ramp DCR or equivalent, at both aggregate and row level  Serve the Gold layer the compliance agent tier reads, including the product-terms and compliance-rule data required for factual claim verification  Build activation integrations to owned channels and paid media, supporting real-time audience push and match-rate monitoring

Platform & Governance  Own Unity Catalog governance: access control, lineage, and tenant isolation across client organisations  Implement PII governance controls at the pipeline layer - redacted s, consent signal propagation, and guardrail validation aligned to GLBA, Fair Lending, UDAAP, and TCPA/CAN-SPAM  Own CI/CD, infrastructure-as-code, cost management, and pipeline observability  Meet the serving contract the agentic layer depends on: a fixed set of signal shapes, queried live on every conversational turn, with predictable latency and per-client isolation

Required Qualifications  5+ years of data engineering experience, including production Databricks work:
Delta Lake, Unity Catalog, Lakeflow/DLT pipelines, structured streaming, and SQL warehouses  Working GCP fluency
- Big Query, Cloud Storage, Cloud Run, IAM  Expert-level SQL and strong Python for transformation logic and data-quality automation  Hands-on experience with at least one clean room technology, with a real understanding of aggregate versus row-level constraints  Deep understanding of identity resolution:…