M26 - Data Engineer

Role Overview

We are seeking an experienced Data Engineer to join a Data Engineering & Infrastructure team supporting a large-scale enterprise data platform.

The platform consolidates workforce and organisational data from multiple source systems to enable analytics, reporting, data sharing, and data-driven decision-making across the organisation.

This is a hands-on, delivery-focused role covering data ingestion, transformation, data quality, data modelling, pipeline operations, and downstream data services.

You will work closely with technical teams, source system owners, business users, and technology partners to maintain and enhance the data platform while ensuring data remains accurate, reliable, traceable, and accessible for approved analytics and reporting requirements.

The current environment primarily uses AWS data services, SQL and Python, with Tableau supporting downstream analytics and visualisation.

• Develop and maintain ETL/data pipelines and file-based interfaces for ingesting data from multiple source systems.
• Perform data profiling, source-to-target mapping, interface specifications, and dataset onboarding.
• Build reliable and maintainable ingestion processes for structured and file-based data sources.

• Develop and maintain SQL and Python processing logic to cleanse, standardise, transform, and reconcile data.
• Work with both current and historical datasets, including complex or legacy data structures.
• Transform source data into clean, reusable, and reliable datasets for downstream consumption.

• Maintain and enhance data models, tables, views, datasets, and dependencies based on evolving business requirements.
• Maintain clear and traceable data mappings, definitions, transformation logic, and processing rules.
• Assess the downstream impact of changes to data structures and transformation processes.

• Develop and automate data validation, reconciliation, and cleansing checks.
• Investigate data anomalies and discrepancies across source, interface, and transformation layers.
• Trace data lineage and identify root causes of data quality issues.
• Implement and validate appropriate fixes and preventive measures.

• Monitor data pipelines, interfaces, and scheduled processing activities.
• Troubleshoot pipeline failures and data processing issues.
• Perform reruns and recovery activities where required.
• Continuously improve pipeline reliability, performance, maintainability, and operational efficiency.

• Support system enhancements and change requests by reviewing requirements and assessing technical impact.
• Develop or modify data processing logic based on approved requirements.
• Execute functional and data validation testing.
• Document defects, investigate issues, and support resolution through implementation.
• Support UAT and production deployment activities where required.

• Work with AWS-based data services, including technologies such as Amazon S3 and Athena.
• Develop and maintain tables, views, queries, and governed datasets for approved reporting and analytics requirements.
• Support Tableau data sources and workbooks where required.
• Enable reliable and controlled consumption of enterprise data by downstream users and applications.

• Translate business and stakeholder requirements into data mappings, transformation rules, and practical technical solutions.
• Collaborate with business users, technical teams, source system owners, and external technology partners.
• Maintain technical documentation, data mappings, processing rules, runbooks, and operational procedures.
• Communicate technical findings, dependencies, risks, and issues clearly to relevant stakeholders.

• Follow established data governance, security, access control, and change management practices.
• Support audit, access review, and assurance activities by providing required technical information and evidence.
• Assist with technical remediation activities arising from governance or assurance reviews.

• Strong proficiency in SQL and working proficiency in Python.
• Hands-on experience developing ETL/data pipelines, data transformations, validation, and automation.
• Experience working with cloud-based data platforms.
• Experience with AWS data services such as Amazon S3, Amazon Athena,…