Senior Databricks Tech Lead​/Architect; Hands-On Onsite

Senior Databricks Tech Lead / Architect

We are seeking a Senior Databricks Tech Lead / Architect for an onsite engagement. This is NOT a paper architect role. The ideal candidate must be a hands-on coding practitioner who can architect scalable data solutions, write and review production-quality code, and provide technical direction to an offshore development team. You will be the primary technical authority for our Databricks platform and own both delivery and design quality.

Key Responsibilities

• Design and implement end-to-end data lakehouse solutions on Databricks (Delta Lake, Unity Catalog, Workflows)
• Write hands-on production-quality PySpark, Python, and SQL code — not just review or approve
• Architect medallion (Bronze/Silver/Gold) data pipelines and ensure they meet performance and reliability standards
• Lead and direct offshore/nearshore development teams: daily standups, code reviews, sprint planning, and technical mentorship
• Define and enforce coding standards, branching strategies, and CI/CD practices for the data engineering team
• Collaborate with business stakeholders, data architects, and product owners to translate requirements into technical designs
• Optimize Spark jobs, cluster configurations, and cost management on Azure/AWS/GCP cloud environments
• Integrate Databricks pipelines with upstream/downstream systems including Kafka, ADF, Airflow, dbt, and REST APIs
• Drive best practices in data quality, data governance, and metadata management
• Proactively identify technical debt and lead remediation efforts
• Serve as the go-to escalation point for all Databricks technical issues within the engagement

Required Qualifications

• 8+ years of overall data engineering experience; 4+ years specifically on Databricks platform
• Deep hands-on expertise with PySpark, Spark SQL, Delta Lake, and Databricks Workflows
• Proven experience architecting large-scale data pipelines in a cloud environment (Azure preferred; AWS or GCP acceptable)
• Demonstrated experience leading and directing offshore or distributed engineering teams
• Proficiency in Python and SQL — must be able to produce and review code, not just advise
• Strong grasp of data modeling, dimensional modeling, and lakehouse design patterns
• Experience with CI/CD tools (Azure Dev Ops, Git Hub Actions) and infrastructure-as-code (Terraform, ARM)
• Databricks Certified Associate Developer or Professional certification is a strong plus
• Excellent communication and stakeholder management skills for both technical and business audiences