Data Engineer; Intermediate

Overview

Hybrid Position. We are looking for a motivated and technically skilled Data Engineer who is eager to learn and grow. This role involves working with modern data platforms and development practices to design, build, and maintain scalable data solutions. Pinelands, Cape Town. Hybrid Position.

• Build and optimize data pipelines and workflows on Databricks and Spark.
• Contribute to the design and implementation end-to-end data flows and CI/CD processes.
• Collaborate with cross-functional teams to deliver reliable, scalable, and high-quality data solutions.
• Continuously expand your knowledge of modern data engineering practices.

• Technical Expertise (3 - 5 years experience)
• Previous experience as a data engineer.
• Proven experience with big data tools, particularly Databricks.
• Strong background in cluster computing using Apache Spark.
• Experience developing and maintaining CI/CD pipelines.
• Proficiency in Python with mandatory PySpark experience.

• Prior work as a backend developer, showcasing strong software engineering fundamentals.
• Familiarity with Azure cloud infrastructure and services/
• Core
  
  Skills:
• Strong communication and collaboration abilities.
• Attention to detail with a focus on data quality.
• Experience in end-to-end data flow design and implementation.
• Familiarity with agile ways of working.
• Growth Mindset. Demonstrated eagerness to learn and adapt, supported by examples of applying new skills in practice.

• Being South Africa’s largest company of its kind – IDM is continuously growing, with no signs of stopping.
• One of our core values is ‘growing people’ and we take that very seriously. We have internal learning opportunities to better our people, and leaders who are experts in their fields and freely impart knowledge and are open to new ideas.
• Our culture is one of collaboration, diversity, and helpfulness. That goes for our people internally and how we deal with our clients.
• At IDM, you can play a part in educating and assisting real South Africans get out of tough financial situations. You can make a difference with the work you do.

• We make sure our employees are mentally and physically healthy with Healthy Together campaigns throughout the year.
• We offer various rewards and awards as tokens of recognition for hard work.
• We have a vibrant office with easy access to stores and outlets if you need.
• We offer company benefits – like a half day off on your birthday, medical aid, retirement annuity, and further study opportunities should you meet relevant requirements.
• We have a culture of growth, meaning you have access to move both upwards or across in the business if you want to.