Principal Data & AI Platform Architect – Azure Databricks

Tampa, Florida, United States

Reporting to the Senior Director of Data & AI, the Senior Data Architect is a hands-on technical leader responsible for the architecture, engineering, governance, and optimization of UVVC’s enterprise data platform. This role will define technical standards and design scalable data pipelines, medallion data products, semantic models, and governed analytics solutions that enable trusted self-service analytics and AI-driven insights across UVVC.

• Design, build, and optimize batch and streaming ETL/ELT pipelines and reusable ingestion frameworks using Azure Data Factory and Databricks across APIs, databases, SaaS platforms, and internal systems.
• Build scalable Delta Lake transformation frameworks using medallion architecture, Spark, and SQL.
• Implement CI/CD, parameterization, triggers, and pipeline automation best practices.
• Architect, manage, and optimize enterprise data environments across Azure Data Lake Storage Gen2 (ADLS Gen2), Azure SQL, and Databricks, including serverless and classic compute strategies, cost governance, and workload isolation strategies.
• Implement Data Ops practices including testing, version control, monitoring, and documentation.

• Design and administer the enterprise Unity Catalog structure, including catalogs, schemas, external locations, storage credentials, groups, service principals, and ownership models.
• Implement least-privilege access, governed tags, attribute-based access-control policies, row-level filters, and column-level masking for PHI, PII, financial, and other sensitive information.
• Establish standards for data classification, lineage, auditability, stewardship, retention, certification, and access reviews.
• Partner with Security, Compliance, Privacy, and business data owners to ensure data solutions align with HIPAA and organizational security requirements.
• Govern tables, views, volumes, functions, metric views, dashboards, models, and Genie Agents through Unity Catalog.
• Design and develop Databricks AI/BI Dashboards and domain-specific Genie Agents for clinical, operational, financial, RCM, marketing, and executive use cases.
• Configure trusted datasets, joins, business terminology, instructions, example queries, dimensions, measures, synonyms, and approved KPI definitions.
• Develop and govern Unity Catalog metric views and semantic definitions to ensure consistent reporting across Databricks AI/BI and Power BI.
• Establish testing and monitoring processes for Genie Agent accuracy, data grounding, security, explainability, performance, and user adoption.
• Ensure that AI-generated results respect Unity Catalog permissions and approved business definitions.
• Design and implement enterprise-grade Databricks Lakehouse Medallion architecture (Bronze, Silver, Gold layers).
• Define and enforce data engineering standards, naming conventions, and architectural patterns across all pipelines.
• Lead the architecture of Delta Lake design patterns, including partitioning, optimization, and data lifecycle management.
• Establish scalable serverless and classic compute strategies, job orchestration frameworks, and workspace organization.
• Evaluate and implement new Databricks capabilities and ensure alignment with enterprise data strategy.

• Work closely with clinical, sales, marketing, finance, RCM, operations, HR, and IT teams to understand business needs.
• Provide technical guidance on data engineering patterns and platform capabilities.
• Clearly communicate progress, risks, and technical decisions to data stakeholders and leadership.

• Develop conceptual, logical, dimensional, and physical data models supporting clinical, operational, financial, marketing, RCM, and workforce analytics.
• Establish conformed dimensions, master and reference data standards, governed KPIs, and reusable semantic definitions.
• Reduce conflicting calculations and duplicate business logic across Databricks AI/BI, Power BI, and downstream applications.

• Establish data-quality rules, reconciliation controls, data SLAs, pipeline observability, alerting, incident response, and root-cause-analysis processes.
• Define standards for schema evolution, change data capture, late-arriving data, historical tracking, retries, and recovery.

• Monitor and optimize Databricks consumption using system tables, workload tagging, budget controls, query profiling, serverless and classic compute selection, SQL warehouse configuration, and data-layout optimization.
• Establish cost allocation and accountability by environment, data product, department, and workload.

• 8+ years of progressive experience in data engineering, data architecture, analytics engineering, or cloud data platforms, including demonstrated ownership of production Databricks architecture.
• Demonstrated technical leadership and mentoring experience within data engineering or…