Fabric Data Engineer — Workplace Engineering

About the Role

Vanguard is standing up Microsoft Fabric as the enterprise data and analytics foundation that powers our Workplace AI, Power BI, and cross-cloud analytics estate. This role partners with Microsoft on a Fabric Enablement engagement and integrates with Vanguard’s data, identity, and security stack, including One Lake Direct Lake against AWS S3, Entra  Okta federation, and Microsoft Purview.

We are hiring a hands-on Fabric Data Engineer to own the data layer of the capability. This builder role designs and implements scalable data products in One Lake — lake houses, warehouses, pipelines, notebooks, and semantic-model-ready Delta tables — and is accountable for the lifecycle, governance, and operational health of the Fabric platform. The complementary AI Engineer builds agents, copilots, and Foundry orchestrations;

this engineer ensures the data underneath is governed, monitored, and ready. You will partner with the AI Engineer on AI-ready data products and semantic-layer handoffs, with the Technical Project Manager on program delivery and change management, and with the Cloud Domain Architect on platform alignment. You will work with the Microsoft CDAO Fabric Enablement team and Vanguard partners across CDAO and Workplace Engineering as a core member of the Workplace AI Fusion Team.

This is a strategic engineering and implementation role, not a support position.

• Design and implement scalable data storage in One Lake using Lake houses (Delta) and Warehouses (T-SQL); select the right storage pattern for each workload and configure SQL analytics endpoints, shortcuts, and One Lake security.
• Build and maintain Spark notebooks (PySpark), Data Factory pipelines, Dataflows Gen2, Copy Jobs, and mirroring for batch and incremental ingestion at enterprise scale.
• Develop Real-Time Intelligence solutions:
  Event streams, Event houses / KQL databases, Activator reflexes, and Spark structured streaming for low-latency workloads.
• Optimize Lakehouse tables (OPTIMIZE, V-Order, Z-Order, partitioning) and Direct Lake semantic-model-ready datasets so downstream Power BI and AI agents perform predictably.

• Implement source control, branching, and CI/CD using native Fabric Git integration (Azure Dev Ops and Git Hub), Fabric Deployment Pipelines, and the Microsoft fabric-cicd Python library.
• Automate Dev / Test / Prod promotion against the Fabric REST API using service principals and Workload Identity Federation; codify environment-aware bindings via Variable Libraries and parameter.yml.
• Operate a Feature → Dev → UAT → Prod branching pattern with mandatory PR review, cherry-pick promotion, and one repo per team to scope blast radius.
• Own the lifecycle of Fabric data components from creation through retirement, ensuring every environment is reproducible from the Git Hub pipeline rather than from the Fabric UI.

• Operate the Fabric F256 capacity: monitor CU consumption with the Capacity Metrics App, manage smoothing windows, diagnose interactive and background throttling, and right-size workloads.
• Build telemetry using the Monitoring Hub, per-workspace Eventhouse-based KQL logs, Eventhouse monitoring, and the Admin Monitoring Workspace to surface refresh failures, pipeline errors, and semantic-model health.
• Define dashboards and alerts for ingestion, transformation, refresh, and capacity health; perform root-cause analysis on production incidents and feed lessons back into platform standards.
• Define and operate the on-call model for production data pipelines and Fabric items in partnership with Tier 3 Engineering.

• Define and enforce Fabric platform standards through Terraform-based IaC using the microsoft/fabric provider (work spaces, capacities, domains, items), workspace templates, naming and tagging conventions, and automated CI policy checks against the Fabric REST API.
• Manage tenant settings, domains, and capacity allocation in partnership with the Fabric Center of Excellence; align identity with Entra  Okta federation; rotate service principals and use PIM for…