Intermediate Data Engineer

We’re working with an established technology consultancy, and they’re looking for an Intermediate Data Engineer to build and support scalable, repeatable and secure data pipelines as part of an agile team.

You’ll build the infrastructure that turns raw data into insight — handling large volumes in batch and real time, integrating diverse sources, and using modern big-data and cloud technologies.

• Build and support scalable, secure data pipelines, integrating diverse data sources
• Handle large volumes of data in batch and real time
• Create data marts using Talend or a similar ETL tool, and manipulate data using Python
• Process data using the Hadoop paradigm (e.g. EMR), and create data feeds into AWS Cloud
• Develop for big data and business intelligence, including automated testing and deployment
• Translate functional and technical requirements into performant solutions

• 3–5 years’ data-engineering or software-engineering experience, including big-data and ETL work
• Strong Python, and PySpark or Spark
• Big-data batch and streaming tools, and AWS (EMR, EC2, S3)
• Data modelling and data-warehousing experience
• A bachelor’s degree in Computer Science, Computer Engineering or equivalent

• AWS certification
• Big Query, DBT and a combination of AWS / GCP / Azure
• AI tooling (e.g. Bedrock or similar)

Hybrid in Johannesburg — three days a week in the office.