Senior Azure Databricks Developer

As a Senior Azure Databricks Developer, you will:

• Design, develop, and maintain scalable data processing solutions using Azure Databricks.
• Build and optimize batch and streaming data pipelines using Python, PySpark, and SQL.
• Design and implement robust data models within the Databricks ecosystem.
• Manage multiple parallel data processing workloads on shared datasets while ensuring performance and reliability.
• Collaborate with cross-functional teams to deliver high-quality data solutions aligned with business requirements.
• Implement CI/CD pipelines using Azure Dev Ops and YAML-based deployment frameworks.
• Utilize Infrastructure as Code (ARM/Bicep) to automate cloud resource provisioning and management.
• Participate in Agile ceremonies and contribute to continuous improvement initiatives.
• Ensure solution quality, maintainability, scalability, and adherence to engineering best practices.
• Make informed technical decisions within a complex and evolving cloud data architecture.

• 6–8 years of overall experience in software engineering, data engineering, or cloud data platform development.
• Strong hands‑on experience with Azure Databricks as a primary development platform.
• Deep expertise in Python, PySpark, and SQL for large-scale data processing.
• Proven experience building and managing both streaming and batch processing solutions.
• Solid experience with Azure cloud services and CI/CD implementation using YAML pipelines.
• Hands‑on experience with Infrastructure as Code tools, particularly ARM Templates and/or Bicep.
• Strong understanding of distributed data processing concepts and performance optimization.
• Experience working in Agile development environments.
• Exposure to end‑to‑end data platform architectures is an added advantage.

• Design reliable and scalable data processing frameworks within Azure Databricks.
• Handle complex data transformation and processing requirements efficiently.
• Optimize data workloads for performance, scalability, and cost‑effectiveness.
• Manage concurrent processing pipelines and shared data dependencies.
• Identify, troubleshoot, and resolve technical issues proactively.
• Communicate technical concepts effectively to stakeholders and team members.
• Balance engineering best practices with practical business requirements.
• Drive continuous improvement initiatives and reduce technical debt.
• Deliver maintainable, production‑grade solutions with a strong focus on quality.

• Opportunity to work on large‑scale cloud‑native data platforms and modern data engineering initiatives.
• Exposure to cutting‑edge Azure and Databricks technologies.
• Collaborative Agile work environment focused on innovation and continuous learning.
• Challenging technical projects that encourage ownership and professional growth.
• A culture that values quality engineering, pragmatic problem‑solving, and teamwork.
• Opportunities to contribute to architecture decisions and influence technical direction.
• Dynamic and supportive team environment focused on delivering impactful solutions.

Want to discuss this opportunity in more detail? Feel free to reach out.