Azure Platform Infrastructure Engineer - Databricks, Python

Overview

Role focuses on designing and implementing data infrastructure and pipelines in Microsoft Azure, with emphasis on Databricks, Data Factory, Data Lake Storage, and Synapse. The role requires strong Dev Ops practices, data modeling knowledge, and awareness of data privacy and compliance.

• Solid understanding of Azure infrastructure (subscriptions, resource groups, resources, access control with RBAC, Azure AD integrations, network concepts, password credential, key management and data protection).
• Strong hands-on knowledge of Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse Serverless Dedicated Spark pools, Python, PySpark and T-SQL.
• Experience designing and developing scripts for ETL processes and automation in Azure Data Factory and Azure Databricks.
• High proficiency in Git, Jenkins and Dev Ops processes to maintain and resolve issues with data pipelines in production.
• Knowledge of implementing Azure technologies and networking via Terraform and the ability to fix issues with Azure infrastructure in production.
• Good understanding of data modeling, Data mart, Data Lakehouse architecture, Slowly Changing Dimensions (SCD), data mesh and delta lake.
• Solid understanding of data privacy and compliance regulations and best practices for preserving customer data.

• Microsoft Azure
• Databricks

• Solid understanding of Azure infrastructure (subscriptions, resource groups, resources, access control with RBAC, Azure AD integrations, network concepts, password credential, key management and data protection).
• Strong hands-on knowledge of Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse Serverless Dedicated Spark pools, Python, PySpark and T-SQL.
• Experience designing and developing scripts for ETL processes and automation in Azure Data Factory and Azure Databricks.
• High proficiency in Git, Jenkins and Dev Ops processes to maintain and resolve issues with data pipelines in production.
• Knowledge of implementing Azure technologies and networking via Terraform and the ability to fix issues with Azure infrastructure in production.
• Good understanding of data modeling, Data mart, Data Lakehouse architecture, Slowly Changing Dimensions (SCD), data mesh and delta lake.
• Solid understanding of data privacy and compliance regulations and best practices for preserving customer data.

6-8 years