Senior Azure Data Engineer

We are looking for a Senior Azure Data Engineer to join our dynamic and forward-thinking data team. The ideal candidate will be responsible for designing, developing, and maintaining robust, scalable, and high-performance data pipelines and architectures using Microsoft Azure technologies. This role requires a deep understanding of cloud-based data processing systems and close collaboration with data scientists, analysts, and business stakeholders to ensure efficient data workflows and reliable data availability.

Key Responsibilities:

• Design and implement scalable and secure data pipelines and ETL processes using Azure Data Factory, Azure Synapse, and Databricks.
• Develop and maintain data lakes and data warehouses on Azure Data Lake Storage.
• Leverage Python and PySpark to manipulate, clean, and transform large-scale datasets.
• Design and optimize SQL queries and develop efficient data models to support analytical and reporting needs.
• Ensure high data quality and integrity through data validation, monitoring, and logging mechanisms.
• Collaborate with cross-functional teams including data scientists, BI developers, and business analysts to understand data requirements.
• Apply Dev Ops practices such as CI/CD pipelines, version control (e.g., Git), and environment management for deployment and integration.
• Monitor, troubleshoot, and optimize data workflows to ensure performance and reliability in a production environment.

Required Skills &

Qualifications:

• 5+ years of experience in data engineering or related roles.
• 2+ years of hands-on experience with Microsoft Azure cloud technologies.

Deep expertise in:

• Strong programming skills in Python and PySpark for big data processing.
• Advanced proficiency in SQL and data modeling principles.
• Solid knowledge of cloud architecture, data warehousing, and distributed systems.
• Strong problem-solving skills and the ability to troubleshoot complex data issues.

Preferred

Skills:

• Experience with Dev Ops practices, CI/CD pipelines, and tools such as Azure Dev Ops or Git Hub Actions.
• Knowledge of data security, data governance, and compliance best practices.
• Familiarity with Delta Lake, Apache Spark, and real-time data processing.
• Experience working in Agile/Scrum environments.

Certifications (Preferred but not required):

• Microsoft Certified:
  Azure Solutions Architect Expert
• Any relevant Azure or data engineering certifications