Data Engineer

We are looking for an experienced Data Engineer to join the Data & AI team delivering core data platform capabilities. This is a hands‑on engineering role focused on designing and building robust Python-based data pipelines, ETL processes, and data models across a modern data platform. The ideal candidate is a strong Python developer with solid SQL skills and a good understanding of data lake architecture.

Exposure to data quality tooling / techniques, whether Informatica, Microsoft Purview, or custom-built frameworks, is a valuable plus.

• Design, build, and maintain end-to-end data pipelines using Python, ensuring reliable data ingestion, transformation, and delivery across the platform.
• Develop and optimise ETL/ELT processes to move and transform data across bronze, silver, and gold layers following the Medallion architecture pattern.
• Write clean, modular, production-grade Python code for data processing, orchestration, and automation tasks.
• Supports in design and implementation of data models, schemas, and storage strategies to support downstream analytics and reporting requirements.
• Build and maintain SQL-based transformations, stored procedures, and views for data validation and reconciliation.
• Develop and manage data ingestion frameworks, handling a variety of source formats (flat files, APIs, databases, streaming).
• Implement data quality checks and validations within pipelines, applying rules across completeness, validity, consistency, uniqueness, accuracy, and timeliness dimensions.
• Monitor pipeline health, build alerting mechanisms, and troubleshoot data issues in production environments.
• Contribute to CI/CD pipelines for data workloads, including automated testing, deployment, and version control practices.
• Produce clear technical documentation for pipelines, data models, and operational runbooks.

• Strong Python development skills with hands‑on experience building production data pipelines (pandas, PySpark, or equivalent).
• Solid SQL skills for complex queries, data transformations, and performance tuning.
• Experience designing and implementing ETL/ELT processes at scale.
• Good understanding of the Medallion architecture (Bronze / Silver / Gold) and modern data lake/lakehouse design patterns.
• Experience with data orchestration tools.
• Working knowledge of cloud data platforms (Azure, AWS, or GCP).
• Familiarity with relational and non‑relational databases.
• Strong problem‑solving skills with the ability to debug complex data pipeline issues.