Azure Databricks Data Engineering Lead

About the job Azure Databricks Data Engineering Lead

Job Description

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.
• Lead large-scale, complex, cross-functional projects to build the technical roadmap for the WFM Data Services platform.
• Lead and review design artifacts.
• Build and own the automation and monitoring frameworks that showcase reliable, accurate, easy-to-understand metrics and operational KPIs to stakeholders for data pipeline quality.
• Execute proof of concept on new technology and tools to pick the best tools and solutions.
• Support business objectives by collaborating with business partners to identify opportunities and drive resolution.
• Communicate status and issues to Senior Starbucks leadership and stakeholders.
• Direct project team and cross-functional teams on all technical aspects of the projects.
• Lead with engineering team to build and support real-time, highly available data, data pipeline and technology capabilities.
• Translate strategic requirements into business requirements to ensure solutions meet business needs.
• Define and implement data retention policies and procedures.
• Define and implement data governance policies and procedures.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re‑designing infrastructure for greater scalability.
• Enable team to pursue insights and applied breakthroughs, while also driving the solutions to Starbucks scale.
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of structured and unstructured data sources using big data technologies.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data‑related technical issues and support their data infrastructure needs.
• Perform root cause analysis to identify permanent resolutions to software or business process issues.

MUST HAVE:

• 7+ years of experience in Databricks, Microsoft Azure, PySpark and Python.