AWS Data Engineer

AWS DATA ENGINEER – CONTRACT

Job Title:

AWS Data Engineer

Location:

[JHB/Hybrid]
*** MUST BE LOCAL IN SOUTH AFRICA

Job Type: [Full-Time /Contract]

Job Description:

We are seeking a skilled AWS Data Engineer to join our dynamic team. The ideal candidate will have a strong background in data engineering, with specific expertise in AWS technologies. You will be responsible for designing, developing, and maintaining our data infrastructure, ensuring efficient data processing and storage solutions.

• Design, develop, and maintain scalable data pipelines using AWS services.
• Write efficient and maintainable Python and PySpark code for data processing.
• Implement and manage ETL processes using AWS Glue.
• Develop and optimize SQL queries for data extraction and transformation.
• Utilize AWS Athena for querying and analyzing large datasets.
• Work with AWS Redshift for data warehousing solutions.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Ensure data quality and integrity across various data sources.
• Monitor and troubleshoot data pipeline issues to ensure smooth operations.
• Stay updated with the latest industry trends and best practices in data engineering and AWS technologies.

• Must be strong in MDM knowledge:
  Experience with building or integrating MDM systems — e.g., designing golden records, matching/merging, data mastering, data stewardship workflows
• Python:
  Proficient in writing Python code for data processing and automation.
• Spark:
  Strong understanding of Apache Spark and experience with PySpark.
• AWS Data Stack:
  Glue, S3, Lambda, Redshift, Athena, Step Functions
• SQL:
  Advanced skills in SQL, with the ability to write complex queries.
• AWS Athena:
  Experience with AWS Athena for querying large datasets.
• Redshift:
  Familiarity with AWS Redshift, including data warehousing concepts and usage.
• Data Warehousing:
  Solid understanding of data warehousing principles and best practices.

• Experience with other AWS services such as S3, Lambda, and Cloud Formation.
• Knowledge of data modeling and schema design.
• Familiarity with data governance and security best practices.
• Experience with version control systems like Git.
• Strong problem-solving skills and attention to detail.
• Excellent communication and collaboration skills.

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 4+ years of experience in data engineering or a related role.