Databricks AI & Governance

The Databricks AI & Governance role will design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows across enterprise data platforms. The role requires strong expertise in data management, governance, AI/ML, Generative AI, Agentic AI, Python, modern AI frameworks, cloud platforms, and governance technologies. The engineer will collaborate with Data Governance Engineers, Data Engineers, Architects, business stakeholders, and client teams to identify governance use cases, develop autonomous and human-in-the-loop agents, integrate enterprise data and governance platforms, and deliver secure, compliant, transparent, and responsible AI solutions.

This is a hybrid role with work locations in Dallas, Chicago, Atlanta, Charlotte, Minnesota, Cupertino, or San Francisco.

• Design, develop, and deploy Agentic AI solutions that automate data governance, data quality, metadata management, lineage analysis, policy enforcement, and stewardship workflows.
• Build scalable AI agents using Large Language Models (LLMs), Retrieval-Augmented Generation (RAG), orchestration frameworks, and enterprise governance tools.
• Partner with Data Governance Engineers, Data Engineers, Architects, and business stakeholders to identify governance use cases and define agent capabilities aligned with enterprise data strategy.
• Develop and maintain autonomous and human-in-the-loop agents supporting data cataloging, business glossary management, policy compliance validation, data ownership assignment, issue remediation, and governance workflow orchestration.
• Integrate AI agents with data catalogs, metadata repositories, data quality tools, MDM platforms, cloud data platforms, and governance technologies including Ataccama, Collibra, Informatica, Microsoft Purview, Databricks, Snowflake, AWS, and Azure.
• Create proof-of-value demonstrations and pilot implementations to validate governance automation opportunities and accelerate stakeholder adoption.
• Collaborate with internal delivery teams and client stakeholders to refine governance backlogs, prioritize agent development initiatives, and incrementally onboard governance controls.
• Implement monitoring, observability, guardrails, and governance controls for Agentic AI solutions to support security, compliance, transparency, and responsible AI practices.
• Support proposal development, solution architecture discussions, effort estimation, and client presentations related to Data and AI governance modernization.
• Contribute to reusable frameworks, accelerators, and best practices for enterprise-scale Agentic AI implementations focused on data governance and management.

• 8+ years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture.
• 5+ years of experience designing and implementing enterprise data governance, Master Data Management (MDM), metadata management, data quality, or regulatory compliance solutions.
• 3+ years of experience developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks.
• 3+ years of hands-on experience with Python and modern AI development frameworks such as Lang Chain, Lang Graph, Semantic Kernel, Auto Gen, CrewAI, OpenAI, Azure AI Foundry, Amazon Bedrock, or equivalent platforms.
• 3+ years of experience working with cloud platforms including Azure, AWS, or Google Cloud Platform.
• 2+ years of experience integrating governance platforms such as Collibra, Databricks Unity Catalog, or related data management technologies.
• 2+ years of experience designing APIs, workflow orchestration, event-driven architectures, or enterprise system integrations.
• Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities.
• Experience working in Agile delivery environments with backlog-driven development, iterative releases, and stakeholder feedback cycles.
• Strong client-facing communication skills with the ability to lead workshops, requirements-gathering sessions, architecture reviews, and executive demonstrations.