More jobs:
Job Description & How to Apply Below
We are seeking a skilled Data Engineer to design, build, and optimize Azure-based data pipelines and lakehouse solutions. The ideal candidate will deliver secure, reliable datasets with strong governance, automation, and documentation while collaborating across teams in an Agile environment. This role requires strong expertise in Databricks, Python, Spark, data pipelines, and Azure ecosystem, along with a focus on performance, scalability, and data quality.
Key Responsibilities
Design and develop scalable data pipelines using Azure and Databricks
Build and maintain ETL/ELT workflows from diverse data sources
Experience with Unity Catalog migration and data governance.
Develop data transformations using Python, Spark, and Pandas
Implement Delta Lake and Medallion architecture (Bronze/Silver/Gold layers)
Manage data storage using Azure Data Lake Storage (ADLS)
Design and optimize SQL queries and data models (star/dimensional schema)
Implement CI/CD pipelines for data workflows using Azure Dev Ops or Git Hub
Ensure data quality, validation, monitoring, and observability
Implement secure data access using Azure Entra (SSO/RBAC)
Maintain clear documentation for pipelines and data contracts
Participate in Agile ceremonies, code reviews, and team collaboration
Support and mentor junior team members
Required Skills & Experience
8+ years of experience in Data Engineering
Strong experience with Azure Databricks (Python, Spark, Pandas)
Experience with Azure Data Factory (ADF) for orchestration
Hands-on experience with Delta Lake and Medallion architecture
Strong knowledge of Azure Data Lake Storage (ADLS)
Experience with Unity Catalog migration and data governance
Strong proficiency in SQL and performance optimization
Experience in ETL/ELT and data modeling (star/dimensional schema)
Experience with CI/CD tools (Azure Dev Ops or Git Hub Enterprise)
Understanding of Azure Entra security and access control
Experience in data validation, testing, and monitoring pipelines
Familiarity with Agile/Scrum methodologies (Jira/Confluence)
Strong communication, documentation, and collaboration skills
Preferred Experience :
Experience with Databricks Dev Ops (cluster configuration, secrets management)
Experience with Azure Functions (Python or C#)
Exposure to Synapse SQL pools, dbt, or Delta Live Tables
Experience in Financial Services domain
Ability to work independently and contribute to team outcomes
Strong focus on secure, production-grade data solutions
Commitment to data reliability, performance, and governance
Continuous learning mindset and adaptability
Responsible use of Generative AI tools with validation and guardrails
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×