Job Description & How to Apply Below
About the Role
We are seeking an experienced Data Engineer Lead to design, develop, and maintain scalable data solutions on Azure and Databricks as part of our enterprise data modernization initiatives. The ideal candidate will have a strong background in data pipeline development, data integration frameworks , and cloud-based data engineering , with deep expertise in tools such as Databricks, Azure Data Factory, Alteryx, Ab Initio, Talend, and Informatica .
This role will lead the design and delivery of high-performance, governed data architectures for large-scale enterprise clients, driving data reliability, compliance, and analytics readiness.
Key Responsibilities
Design and implement scalable and reusable data pipelines using Databricks (PySpark, Delta Lake) and Azure Data Factory , ensuring high throughput and low latency.
Develop and maintain Common Data Platforms (CDP) and Medallion Architecture-based data lakes to consolidate legacy systems and improve data availability.
Create and manage backend data services using Python, Spark, or C# , ensuring performance, reliability, and scalability.
Develop data ingestion frameworks using metadata-driven and schema-drift–handling designs to support multi-source data integration.
Lead data integration and migration projects using Talend, Ab Initio, Alteryx, and Informatica , modernizing traditional ETL workflows into cloud-native pipelines.
Implement data quality, lineage, and governance frameworks (Unity Catalog, Data Stewardship, Data Catalog, MDM).
Design and maintain GDPR/CCPA-compliant data processing and ensure adherence to global data privacy standards.
Build and optimize Feature Store Data Marts to support AI/ML analytics teams and downstream BI consumers.
Collaborate with cross-functional teams (Data Scientists, BI Developers, Product Owners) to define and deliver data-driven business outcomes.
Mentor junior engineers in best practices of data engineering , CI/CD, and infrastructure-as-code deployments.
Utilize Redis and caching mechanisms to optimize performance and reduce load latency in analytic workloads.
Take end-to-end ownership of assigned modules ensuring timely delivery, documentation, and performance optimization.
Technical Environment
Cloud Platforms: Microsoft Azure (Data Factory, Synapse, Data Lake, Databricks), AWS (S3, EC2, Glue, Redshift)
Data Engineering Tools: Databricks (DLT, Auto Loader, Unity Catalog), Talend, Ab Initio, Informatica, Alteryx
Programming
Languages:
Python, PySpark, SQL, Unix Shell Scripting
Data Modeling: Dimensional, Relational, and Medallion architecture
Dev Ops / CI-CD: Git Hub Actions, Azure Dev Ops, Databricks Asset Bundles (DABs)
Data Governance & Security: RBAC/ABAC, Data Stewardship, MDM, GDPR/CCPA compliance
Monitoring & Scheduling: Airflow, Logic Apps, Azure Monitor
Required Qualifications
12+ years of experience in Data Engineering, Data Architecture, or ETL Development .
Proven experience leading Azure + Databricks –based modernization or migration programs.
Deep understanding of data integration tools (Talend, Informatica, Ab Initio, Alteryx) and metadata-driven pipeline design .
Hands-on experience with PySpark, SQL, and Azure Data Services .
Strong understanding of data modeling, data quality, and governance frameworks .
Experience with real-time and batch data ingestion , including Kafka, Kinesis, or Event Hub.
Excellent problem-solving, communication, and stakeholder management skills.
Preferred Qualifications
Bachelor’s degree in computer science, or a related field
Strong analytical and problem-solving skills
Azure Data Engineer Certification or equivalent
Familiarity with AI/ML data preparation and Feature Store design
Experience implementing cost optimization and cluster governance on Databricks
Why Join Us
Be part of a data modernization journey with a global enterprise brand.
Work with cutting-edge cloud and data engineering tools .
Collaborate with a highly skilled, cross-functional team of architects, analysts, and data scientists.
Opportunity to lead and innovate within a large-scale, multi-domain data transformation initiative .
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×