Azure Platform Infrastructure Engineer - Databricks, Python

Job Description

Desirable

Skills:

· Microsoft Azure

· Databricks

Role

Description:

· Solid understanding of Azure infrastructure (subscriptions, resource groups, resources, access control with RBAC, Azure AD integrations, network concepts, password credential, key management and data protection).

· Strong hands-on knowledge of Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse Serverless Dedicated Spark pools, Python, PySpark and T-SQL.

· Experience designing and developing scripts for ETL processes and automation in Azure Data Factory and Azure Databricks.

· High proficiency in Git, Jenkins and Dev Ops processes to maintain and resolve issues with data pipelines in production.

· Knowledge of implementing Azure technologies and networking via Terraform| and the ability to fix issues with Azure infrastructure in production.

· Good understanding of data modeling, Data mart, Data Lakehouse architecture, Slowly Changing Dimensions (SCD), data mesh and delta lake.

· Solid understanding of data privacy and compliance regulations and best practices for preserving customer data.

Experience

Required:

6-8 years

Requirements
Desirable

Skills:

• Microsoft Azure
• Databricks Role

Description:

• Solid understanding of Azure infrastructure (subscriptions, resource groups, resources, access control with RBAC, Azure AD integrations, network concepts, password credential, key management and data protection).
• Strong hands-on knowledge of Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Synapse Serverless Dedicated Spark pools, Python, PySpark and T-SQL.
• Experience designing and developing scripts for ETL processes and automation in Azure Data Factory and Azure Databricks.
• High proficiency in Git, Jenkins and Dev Ops processes to maintain and resolve issues with data pipelines in production.
• Knowledge of implementing Azure technologies and networking via Terraform| and the ability to fix issues with Azure infrastructure in production.
• Good understanding of data modeling, Data mart, Data Lakehouse architecture, SCD, data mesh and delta lake.
• Solid understanding of data privacy and compliance regulations and best practices for preserving customer data. Experience

Required:

6-8 years