Senior Data Engineer

SENIOR DATA ENGINEER

The Senior Data Engineer designs, builds, and owns the enterprise data platform that powers analytics and data-driven decision‑making across the organization's global operations. This is a full-stack data-engineering role spanning the complete Microsoft Fabric medallion architecture (Bronze, Silver, Gold, and Warehouse), covering source-system ingestion and Spark and PySpark transformation through dimensional modeling, the Power BI semantic layer, and Row‑Level Security. The position requires deep, specialized technical expertise, the ability to work independently on highly complex projects with minimal supervision, sound architectural judgment on production systems, and a commitment to comprehensive documentation and governance.

The role partners closely with business stakeholders, internal IT, and external delivery vendors, and sets technical standards and mentors other members of the analytics and engineering team.

• Designs, builds, and maintains end-to-end data pipelines across the Microsoft Fabric medallion architecture (Bronze, Silver, Gold, and Warehouse) using Spark and PySpark notebooks orchestrated through Fabric Data Pipelines.
• Develops and owns ingestion from multiple heterogeneous source systems, including transactional and TMS databases, CosmosDB document stores, and file‑based feeds, applying appropriate replication strategies (Change Data Capture, mirrored or current‑state replication, and watermark‑based incremental loads).
• Implements robust ELT and ETL patterns: incremental and full‑load logic, idempotent MERGE operations, surrogate‑key generation, deduplication, typed‑NULL handling, and runtime parameterization of load identifiers and lakehouse targets.
• Optimizes pipeline performance, compute and capacity usage, and cost, and ensures the platform scales reliably for a global user base.

• Designs dimensional models (star schema, fact and dimension tables, bridge tables, and slowly changing dimensions) in the Fabric Data Warehouse to support analytical and reporting workloads.
• Authors and maintains performant T‑SQL stored procedures, views, and warehouse tables with explicit, well‑documented contracts (explicit column lists, typed columns, and semantic ordering).
• Defines data requirements and business logic for new data products and translates them into sound, maintainable warehouse structures.

• Creates and optimizes Power BI semantic models over the warehouse, including complex DAX measures and calculated columns, time intelligence, and Direct Lake or Import configurations.
• Designs, implements, and troubleshoots Row‑Level Security (RLS) and user access controls across multiple organizational levels and tenants, including embedded scenarios.
• Provides the engineering foundation for interactive dashboards and supports BI developers and analysts consuming the platform.
• Performs comprehensive data reconciliation and validation, identifies and root‑causes discrepancies and anomalies, and remediates defects across the pipeline.
• Owns production incident response for data pipelines and semantic‑model refreshes, diagnosing failures, restoring service, and implementing durable fixes.
• Tests changes thoroughly before deployment to ensure accuracy, performance, and functionality across scenarios and environments.

• Establishes and enforces deployment discipline across Dev, UAT, and Production, including source control (Git), branch protection, promotion rules, and release governance for internal and vendor contributors.
• Creates and maintains comprehensive technical documentation covering data sources, lineage, relationships, DAX measures, business logic, runbooks, and known issues, accessible to both technical and non‑technical audiences.
• Configures monitoring, alerting, and validation routines on critical pipelines and contributes to the team's standards, patterns, and knowledge base.
• Works directly with global business stakeholders to gather requirements and translate them into technical specifications and data products.
• Coordinates with and…