AI Data Ops Engineer

What you’ll own

• Engineer batch and stream pipelines using Fabric Data Pipelines, Azure Data Factory, Synapse or Databricks.
• Implement data quality rules, schema validation, de-duplication (SCD) and reconciliation checks.
• Operationalize lineage, cataloging and classifications with Microsoft Purview and enforce RBAC and access patterns.
• Automate CI/CD via Azure Dev Ops or Git Hub with environment promotion, infrastructure‑as‑code (Bicep/Terraform) and secrets management via Key Vault.
• Build feature stores and model‑serving data contracts in partnership with MLOps and AI engineering teams.
• Own reliability alerts, runbooks, on‑call rotation and cost and performance optimization.
• Review and finalize vendor‑delivered data pipelines and data architecture for AI projects, ensuring compliance with client standards for security, performance and reliability, and approve production readiness.
• Collaborate with delivery partner squads on interface specifications, test data and delivery checkpoints, support SIT, UAT and production cutover.
• Define and enforce Data Contracts and SLAs per priority dataset, schema, refresh frequency, quality thresholds, reconciliation checks and consumer expectations.
• Own data incident management, classification, RCA, corrective actions and prevention of recurring nonconformities.
• Formalize the handshake with AI/ML Dev Ops engineers on feature and embedding pipelines, monitoring hooks and release gates for data‑dependent AI deployments.

• Python and PySpark, SQL, Lakehouse patterns, medallion architecture.
• Microsoft Purview catalog, lineage, classifications, data privacy controls and masking.
• CI/CD with Azure Dev Ops or Git Hub, IaC with Bicep or Terraform, Docker basics.
• Observability:
  Kusto KQL, Azure Monitor, Log Analytics, performance tuning and Fin Ops.
• Production incident response and RCA discipline.

• 6-8 years in data engineering with strong SQL and PySpark and cloud‑native data services.
• Hands‑on experience across the Azure data stack:
  Fabric/Synapse, Data Factory, ADLS Gen2, Delta Lake/Parquet.
• Bachelor’s in Computer Science, Engineering or equivalent.

• Microsoft Certified:
  Azure Data Engineer Associate (DP-203)

• Microsoft Certified:
  Azure Fundamentals (AZ-900)
• Databricks Data Engineer Associate or Professional