Azure Databricks & Agentic AI Architect
Listed on 2026-10-10
-
Software Development
AI Engineer (Applied/Software), Data Engineering
Title:
Azure Databricks & Agentic AI Architect
Location:
Remote
We are looking for a highly experienced Azure Databricks & Agentic AI Architect to design and lead next-generation AI-powered data platforms. The ideal candidate will have 15 years of Data & Analytics experience, strong Azure Databricks/Lakehouse expertise, and hands-on experience with Generative AI, Agentic AI, and AI-driven Data Engineering.
Key ResponsibilitiesDesign enterprise-scale Azure Databricks Lakehouse platforms using Delta Lake, Unity Catalog, ADLS Gen2, Databricks Workflows, and Delta Live Tables.
Lead the implementation of Agentic AI for autonomous data ingestion, mapping, transformation, data quality, metadata enrichment, lineage, testing, and pipeline optimization.
Design AI-driven Data Development Lifecycle (AI-DLC) solutions covering requirements, data modeling, code generation, testing, documentation, deployment, and monitoring.
Architect RAG-based GenAI solutions using enterprise data.
Design AI agent orchestration using Azure OpenAI, Lang Graph, Lang Chain, Semantic Kernel, Auto Gen, and MCP.
Build domain-specific AI agents for Data Engineering and Analytics teams.
Design self-healing data pipelines and AI-driven incident triage, failure prediction, SLA monitoring, capacity planning, and cost optimization.
Establish Human-in-the-Loop (HITL) frameworks and governance for autonomous AI agents.
Define enterprise AI governance covering security, observability, auditability, prompt governance, cost monitoring, and responsible AI.
Mentor architects, data engineers, and AI engineers.
Lead enterprise architecture discussions and communicate AI strategy and business value to senior/CXO stakeholders.
Required Skills15 years of Data & Analytics experience, Strong hands-on Azure Databricks, Lakehouse Architecture, Delta Lake & Unity Catalog, ADLS Gen2, Azure Data Factory, PySpark, Spark SQL, Python & SQL, Azure OpenAI, Generative AI & Agentic AI, RAG Architecture, Lang Chain / Lang Graph, MCP Protocol, AI Agent Orchestration, Vector Databases / Knowledge Graphs, Azure Dev Ops & CI/CD
MLOps / LLMOps, AI Testing & Evaluation Frameworks
Preferred SkillsDatabricks Certified Data Engineer Professional, Databricks Certified Solution Architect, Azure Solutions Architect Expert (AZ-305), Azure Data Engineer (DP-203)
Azure OpenAI / Generative AI certifications, Experience leading enterprise AI transformation initiatives, Strong executive and client-facing communication skills
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).