Databricks Solution Architect

• Lead the end-to-end architecture and solution design for enterprise data platforms on Azure, focusing on Databricks Lakehouse, Delta Lake, and scalable cloud-native data ecosystems
• Define target-state data architecture, ingestion patterns, transformation frameworks, and serving layers for reporting, advanced analytics, ML, and business-critical decisioning
• Design and implement ETL/ELT pipelines using PySpark, SQL, Databricks Workflows, Auto Loader, and Delta Live Tables
• Own architecture standards for data modeling, medallion design, reusable engineering patterns, CI/CD, code quality, environment strategy, and release management
• Drive platform governance and security using Unity Catalog, RBAC/ABAC controls, lineage, auditability, and Azure Purview integration
• Optimize Spark workloads, cluster policies, partitioning, file sizing, caching, and compute costs
• Collaborate with stakeholders, product owners, analysts, architects, and downstream consumers to translate requirements into scalable technical designs
• Provide technical leadership through design reviews, implementation guidance, architectural issue resolution, and best-practice establishment
• Evaluate Databricks capabilities including Photon, serverless compute, Lakehouse Federation, and streaming patterns
• Ensure delivery governance through estimation, technical planning, dependency management, risk mitigation, and Agile execution

• 10–15 years of experience in data engineering, cloud data platform design, or enterprise data architecture
• At least 5+ years of hands-on experience with Databricks and Azure
• Bachelor’s degree in computer science, Information Technology, Engineering, or a related discipline; master’s degree preferred
• Expertise in Lakehouse, medallion architecture, data modeling, data warehousing, and scalable ingestion and transformation frameworks
• Proficiency in Databricks, PySpark, Python, SQL, Delta Lake, Databricks Workflows, Auto Loader, and Delta Live Tables
• Experience with Azure Data Factory, Azure Data Lake Storage, Azure Key Vault, Azure Dev Ops, and enterprise cloud ecosystem integration
• Experience defining architecture standards, reusable design patterns, CI/CD strategy, environment management, and delivery best practices
• Experience with Unity Catalog, role-based access controls, data governance, lineage, security frameworks, and Azure Purview
• Ability to optimize large-scale Spark and Databricks workloads, including performance tuning, cluster sizing, workload management, and cost optimization
• Experience translating business requirements into scalable solution designs and implementation roadmaps
• Strong communication, leadership, and problem-solving skills; ability to mentor teams, review technical designs, and drive architecture decisions
• Insurance domain knowledge is preferred

Demonstrates expertise in designing and implementing enterprise data architectures on Azure, with a focus on Databricks Lakehouse and Delta Lake. Proficient in optimizing data ingestion, transformation frameworks, and ensuring data governance and security.

• Azure Data Platform Design
• Databricks Lakehouse Architecture
• ETL/ELT Pipeline Development
• Data Governance and Security
• Cloud-Native Data Ecosystems

• Databricks
• Py Spark
• SQL
• Delta Lake
• Data Modeling
• Data Warehousing
• CI/CD
• Performance Tuning
• Agile Execution
• Architecture Standards

• Leadership
• Communication
• Problem-Solving
• Mentoring

• Bachelor’s Degree in Computer Science
• Master’s Degree Preferred

• Insurance Domain Knowledge
• Data Governance
• Medallion Architecture
• Scalable Ingestion Frameworks

• Azure Data Factory
• Azure Data Lake Storage
• Azure Key Vault
• Azure Dev Ops
• Unity Catalog
• Azure Purview