Lead Apache Airflow Engineer

Overview

Lead Apache Airflow Engineer to spearhead the development and enhancement of our workflow orchestration platform. In this role, you will drive the evolution of Airflow components, integrate new versions with Cloud Composer and lead a team of engineers in building robust data pipelines. This role requires working from the office in Krakow 3 days per week.

• Develop and maintain Airflow Operators, Hooks and Sensors
• Integrate new Airflow versions with Cloud Composer
• Fix bugs in Airflow and implement improvements
• Implement tests for Airflow, including Unit Testing and Solution Testing
• Create internal documentation for work artifacts
• Lead and mentor a team of Airflow engineers
• Collaborate with Open Source communities to contribute improvements

• 5+ years of experience in Python
• 1+ years of relevant leadership experience
• Programming experience with Cloud APIs (REST/RPC)
• Basic knowledge of Dev Ops/Sys Admin
• Basic understanding of Linux and Kubernetes, including how to build container images
• Familiarity with Cloud platforms
• Expertise in Airflow or similar experience in building workflows (e.g. based on Luigi, Oozie, Argo, Azkaban, Apache HiFi)
• Experience working with Open Source communities
• Communication skills in English, spoken and written (B2+)

• Understanding of Cloud Architecture, including scalability and IaaS principles