Senior Data Engineer; Data Pipelines, ETL​/ELT, AWS

Data engineering here is an integration problem before it is anything else. A Cape Town business whose work runs on regulated fund and market data is hiring a senior data engineer to take technical ownership of it: feeds from a spread of outside vendors and internal platforms, landing across a hybrid on-premises and multi-cloud estate, that have to reconcile, hold up under audit and stay supportable at volume.

The seat is a senior individual contributor one, not a management one, and it is scoped as the step before Lead Data Engineer or Data Architect - deliberately. You stay hands-on in the pipelines every week. You also carry the technical standard for the team: reviewing designs and code, mentoring the engineers around you, and leading the conversations with the business stakeholders whose data this is.

• Lead the design and implementation of complex, large-scale data pipelines integrating market, reference and investment data.
• Solve advanced data engineering problems - high-volume processing, complex transformations, reconciliation and performance optimisation.
• Make sure what you deliver is scalable, resilient, secure and fit for purpose in a regulated financial environment.
• Act as a technical authority in the team, setting the standard for coding, testing, deployment and operational support.
• Review the designs, code and implementations of junior and mid-level engineers for quality and consistency, and contribute to the standards, patterns and reusable frameworks the team works to.
• Coach and mentor junior and mid-level data engineers in the tools, techniques and technology stack, and guide them on solution design and operational readiness.
• Lead technical engagement with business stakeholders, architects and analysts to refine requirements and shape solutions that will actually hold up.
• Translate complex business and regulatory requirements into clear technical designs and delivery plans.
• Work with data and IT architects so what you build lines up with enterprise architecture, integration patterns and data models.
• Make sure pipelines meet data governance, lineage, auditability and regulatory expectations, balancing delivery speed against control and long-term maintainability.
• Own production pipelines end to end, with the monitoring, alerting and support processes that go with them.
• Lead root-cause analysis on complex production issues and put durable fixes in rather than patches.

• A track record of leading the delivery of complex data integration work.
• Modern programming skills - Python, Java or JavaScript - or deep hands-on experience with low-code data integration platforms.
• Advanced experience with ETL and ELT frameworks, orchestration tools and scheduling platforms.
• A solid grasp of hybrid on-premises and cloud-based data architectures, AWS in particular.
• Real depth in data engineering inside complex enterprise environments - built over years rather than months - or equivalent depth carried across from software engineering, or from a business-oriented or data-focused career that took you into engineering.
• Strong understanding of integration patterns, performance optimisation and data quality frameworks.
• A tertiary qualification in computer science, information systems, information technology, engineering or similar.
• A demonstrated ability to mentor and lift less-experienced engineers, and to communicate clearly with technical and non-technical audiences alike.
• A high level of ownership, accountability and attention to detail.

If you are a software engineer looking to move into data engineering, or you moved into engineering from a business or data background and have been building at this level since, this role is open to you on the strength of what you have actually…