Databricks Solution Architect

We are seeking an experienced Databricks Solution Architect to design and implement enterprise-scale data engineering solutions, Lakehouse architectures, and scalable cloud data platforms. The ideal candidate will have strong hands-on expertise in Databricks, Apache Spark, PySpark, Unity Catalog, Delta Lake, and MLflow, along with the ability to lead architecture design and communicate effectively with clients and stakeholders.

• Design and implement enterprise data solutions using Databricks and Lakehouse architecture.
• Develop scalable data pipelines using Apache Spark, PySpark, and Databricks Workflows.
• Implement Medallion Architecture (Bronze, Silver, and Gold layers) for data processing and transformation.
• Configure Unity Catalog for data governance, security, access control, and lineage.
• Implement Delta Lake solutions to ensure data reliability, consistency, and performance.
• Leverage MLflow for experiment tracking, model management, and ML lifecycle operations.
• Define architecture standards, design patterns, and best practices for enterprise data platforms.
• Optimize data processing workloads, pipeline performance, and cloud resource utilization.
• Collaborate with clients, engineering teams, and business stakeholders to translate requirements into technical solutions.
• Provide technical leadership, conduct architecture reviews, and guide development teams.
• Explore and integrate Generative AI capabilities using Mosaic AI, Vector Search, and multi-agent AI frameworks where applicable.

• Strong experience as a Databricks Solution Architect or in a similar data architecture role.
• Expertise in Unity Catalog, Delta Lake, MLflow, and Databricks Workflows.
• Hands-on coding experience with Apache Spark and PySpark.
• Strong understanding of Medallion Architecture and Lakehouse design principles.
• Experience designing enterprise-scale data engineering solutions and scalable data platforms.
• Knowledge of data governance, data quality, security, and performance optimization.
• Excellent solution design, architectural decision-making, client communication, and problem-solving skills.

• Experience with Generative AI, Databricks Mosaic AI, Vector Search, and multi-agent AI systems.
• Hands-on experience with AWS and/or Microsoft Azure.
• Experience in financial services, healthcare, or insurance domains.
• Knowledge of cloud-native architecture, CI/CD, and data platform modernization.