AWS Data Engineer

Overview

Rayo, part of the UBDS Group, is seeking highly skilled, Security Cleared AWS Data Engineers at all levels to design, build, and maintain scalable data pipelines and infrastructure using AWS-native services. This role focuses on data orchestration, automation, and containerisation to support high-performance data workflows across the organisation.

• Develop and maintain data pipelines using Python, SQL, and Apache Airflow.
• Containerise data applications and workflows using Docker.
• Build and optimise data solutions using AWS services including:
• Redshift, Open Search, Lambda, Glue, Step Functions, and Batch.
• Collaborate with cross-functional teams to deliver robust, secure, and scalable data infrastructure.
• Manage version control using Git Lab (or similar).
• Monitor and troubleshoot data pipeline performance and reliability.
• Contribute to documentation and process improvement initiatives.
• Ensure data quality, governance, and security best practices are followed.

• Essential:
• Strong programming skills in Python and proficiency in SQL.
• Experience with Apache Airflow for DAG orchestration and monitoring.
• Hands-on experience with Docker for containerisation.
• Proficient in AWS data services:
  Redshift, Open Search, Lambda, Glue, Step Functions, Batch.
• Familiarity with CI/CD pipelines and YAML-based configuration (e.g., Git Lab CI/CD).
• Proficient in Git and collaborative development using Git Lab (or similar).
• Understanding of AWS security best practices, IAM policies, and RBAC.
• Desirable
  
  Skills:
• Experience with AWS services such as Athena, SQS, Cloud Watch, Cloud Trail, EMR.
• Exposure to infrastructure-as-code tools (e.g., Terraform, Cloud Formation).
• Familiarity with documentation tools like Confluence and README standards.
• Experience working in consulting or client-facing environments.
• Soft Skills:
• Strong problem-solving and troubleshooting abilities.
• Comfortable working in Agile environments.
• Effective stakeholder management and communication skills.
• Collaborative mindset and willingness to share knowledge.