Sr. Technical Lead
Listed on 2026-06-28
-
IT/Tech
Data Engineering, Cloud Computing: Infrastructure & Operations
Sr. Technical Data Lead
Position
Description:
Seeking a Sr. Technical Data Lead who has experience developing and deploying data pipelines and applications on Microsoft Azure as well as exposure to AI/ML concepts. Common tools used are Python, PySpark, SQL, and Databricks. This individual should be able to lead teams and multiple work streams of data engineers, data scientists, and data analysts as well as manage expectations with client stakeholders.
The candidate will work jointly with the cloud architects to understand the overall vision of the solution and guide team members to execute on that vision. At times, this candidate will need to be able to jump in and contribute individually to the build out of the solution.
Your future duties and responsibilities:
- The Sr. Technical Data Lead will be responsible for development, deployment, maintenance and support of data pipelines on the MS Databricks platform
- Will streamline code, rationalize datasets and tables to arrive at single source of truth
- Will build detailed architecture diagrams or Entity Relationship Diagrams for all workflows and processes
- Will lead multiple data engineering work streams towards a common vision
- Will manage stakeholder expectations and act as the liaison between architects and development team
Required qualifications to be successful in this role:
- 4 to 5 years of hands on experience with cloud automation, scripting in an Azure environment, and data transformation
- Understanding of high level AI/ML concepts and in particular experience with feature engineering and natural language processing
- Experience transforming unstructured data (jsons, text, etc.) for inputs in AI/ML models
- Experience in development of apache Spark code using PySpark or Scala
- Experience in using Azure Databricks Platform
- Experience in using Azure Data Factory to call Azure Databricks notebook activities
- Experience in using Azure Dev Ops based repositories
- Good fundamental knowledge about distributed computing, RDBMS and Dimensional modeling concepts
- Exposure to Azure Storage (Blob/Data Lake Store), Azure Catalog, Databricks Delta
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).