Databricks Architect – DBU Optimization

15+ years overall | 6+ years on Databricks

We are seeking a Databricks Architect to design and optimize enterprise‑scale Databricks platforms with a strong focus on DBU cost optimization, performance tuning, and scalable Lakehouse architecture
.

• Architect Databricks solutions for batch, streaming, and analytics workloads
• Drive DBU optimization through cluster strategy, autoscaling, auto‑termination, and Photon adoption
• Design and optimize Delta Lake tables and Spark workloads
• Define best practices for job clusters, cluster policies, and workspace governance
• Implement Unity Catalog for security and data governance
• Partner with cloud and data teams to optimize performance and cost

• Design end‑to‑end Databricks architectures for batch, streaming, and analytics workloads
• Define best practices for cluster strategy, workspace design, and multi‑environment setups (Dev/Test/Prod)
• Strong experience with Databricks (AWS/Azure/GCP)
• Deep knowledge of Spark, Delta Lake, and DBU consumption models
• Hands‑on expertise in PySpark/Scala and SQL
• Analyze Databricks usage patterns and identify DBU cost optimization opportunities
• Define and enforce cluster policies to control DBU consumption

• Job vs all‑purpose clusters
• Autoscaling and auto‑termination
• Experience with cloud data platforms and object storage
• Ability to lead architecture decisions and guide engineering teams

• Databricks certifications
• Experience with large‑scale enterprise data platforms.