Sr. Azure Data Engineer
Listed on 2026-10-15
-
IT/Tech
Data Engineering, Azure
Job Title:
Azure Data Engineer
Location:
Onsite:
Greenfield, IN
Duration: FTE
Key Fields
Inputs
Educational Qualification*
Any Bachelor's Degree
Experience Range
10+ years
Tagline/Tech Stack Snapshot -
Azure Databricks | PySpark | ADF | ADLS Gen2 | Azure SQL | Python | Unity Catalog | Delta Lake | Azure Cloud | CI/CD | Git | Azure Dev Ops
Role Summary - (To be filled by Practice /DO)
We are looking for an experienced Data Engineering Tech Lead with strong expertise in Azure Databricks, PySpark, Azure Data Factory (ADF), ADLS Gen2, Azure SQL, Python, and Azure Cloud. The role will lead the design, development, optimization, and maintenance of scalable enterprise data engineering and Lakehouse solutions, including batch and near-real-time data processing.
Primary (Must have skills)* - To be Screened by TA Team
4+ years of experience in Azure Databricks with PySpark.
2+ years of experience in Databricks workflow & Unity catalog.
3+ years of experience in ADF (Azure Data Factory).
3+ years of experience in ADLS Gen 2.
3+ years of experience in Azure SQL.
5+ years of experience in Azure Cloud platform.
2+ years of experience in Python programming & package builds.
Why This Role Matters - New addition (To be filled by Practice /DO)
The Azure Data Engineering platform is a critical backbone for scalable, secure, and reliable enterprise data processing and analytics.
This role ensures:
Reliable and scalable data pipelines across enterprise data sources
High-performance data processing using Databricks, PySpark, and ADF
Strong data governance, security, and quality across the Lakehouse
Continuous technical leadership, optimization, and modernization of the data platform
What You'll Do/
Job Description of Role* (RNR) - To be Evaluated by Technical Panel (Define it to give more clarity)
Key technical skills :
Data management experience handling Analytics workload covering design, development, and maintainenance of lakehouse solutions sourcing data from platforms such as ERP sources, API sources, Relational stores, No
SQL and on-prem sources using Databricks/PySpark as distributed /big data management service,supporting batch and near-real-time ingestion, transformation, and processing.
Ability to optimize Spark jobs and manage large-scale data processing using RDD/Data Frame APIs.Demonstrated expertise in partitioning strategies, file format optimization (Parquet/Delta), and Spark SQL tuning.
Familiarity with Databricks runtime versions, cluster policies, libraries, and workspace management.
Skilled in governing and manage data access for Azure Data lakehouse with Unity Catalog. Experience in configuring data permissions, object lineage, and access policies with Unity Catalog.
Understanding of integrating Unity Catalog with Azure AD, external metastores, and audit trails.
Experience in building efficient orchestration solutions using Azure data factory, Databricks Workflows.
Ability to design modular, reusable workflows using tasks, triggers, and dependencies.
Skilled in using dynamic expressions, parameterized pipelines, custom activities, and triggers.
Familiarity with integration runtime configurations, pipeline performance tuning, and error handling strategies.
Strong experience in implementing secure, hierarchical namespace-based data lake storage for structured/semi-structured data, aligned to bronze-silver-gold layers with ADLS Gen
2. Hands-on experience with lifecycle policies, access control (RBAC/ACLs), and folder-level security. Understanding of best practices in file partitioning, retention management, and storage performance optimization.
Capable of developing T-SQL queries, stored procedures, and managing metadata layers on Azure SQL.
Comprehensive experience working across the Azure ecosystem, including networking, security, monitoring, and cost management relevant to data engineering workloads.
Understanding of VNets, Private Endpoints, Key Vaults, Managed Identities, and Azure Monitor.
Exposure to Dev Ops tools for deployment automation (e.g., Azure Dev Ops, ARM/Bicep/ Terraform).
Experience in writing modular, testable Python code used in data transformations, utility functions, and packaging reusable components.
Familiarity with Python environments, dependency management (pip/Poetry/Conda), and packaging libraries.
Ability to write unit tests using PyTest/unittest and integrate with CI/CD pipelines.
Lead solution design discussions, mentor junior engineers, and ensure adherence to coding guidelines, design patterns, and peer review processes.
Able to prepare…
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).