Senior Databricks Engineer

Position Summary The Senior Data Engineer
- Databricks will own the end-to-end data lifecycle, from analysis and solution design through implementation, orchestration, governance, and optimization. This role will build modern data solutions on the Databricks Data Intelligence Platform across Microsoft Azure, AWS, or GCP, using Databricks, Python, PySpark, and SQL to deliver scalable ingestion, transformation, modeling, and pipeline orchestration. As a senior engineer, you will lead solution design, establish engineering standards, and guide other data engineers in building reliable, governed, and production-ready data pipelines that support analytics, reporting, and AI/ML use cases.

• Design, develop, and maintain robust batch and streaming data pipelines for extracting, transforming, and loading data from multiple sources into the Databricks lakehouse on Azure.
• Automate and streamline data ingestion using Auto Loader, Lakeflow Connect, and Azure services such as Azure Data Factory and Event Hubs.
• Build and manage pipelines with Lakeflow Spark Declarative Pipelines, formerly Delta Live Tables, including data quality expectations