Senior Data Platform Engineer

As the Senior Data Platform Engineer in the Cyber Data Platform team, you will play a key role in building and maintaining a robust data platform that enables users to develop advanced analytics, machine learning, and GenAI solutions to strengthen our security defences. You will collaborate closely with cross functional data teams to ensure the platform is scalable, secure, and aligned with cybersecurity goals.

Also, you will contribute to the data platform architecture and mentor junior team members to deliver high-quality data services.

Design, build and management of realtime, near realtime and batch data architectures that support threat detection, incident response and reporting through advanced analytics, machine learning and GenAI capabilities.

Implement and manage the automated frameworks for integrating data from various security sources into the security data lake. Also, the data transformation frameworks to move data between the raw, trusted, and curated data layers.

Contribute to raising the quality bar of the team's codebase by producing high-quality code, conducting thorough peer reviews, and proactively providing constructive feedback to other team members on their code.

Implement various automations and Dev Ops practices to streamline the deployment, configuration, and management of data platform components.

Collaborate with teams from various departments, effectively communicating technical concepts and solutions to individuals with different levels of technical expertise.

• Solid programming experience with programming languages like Python along with proficiency in SQL
• Experience building data platforms on cloud services like Databricks on Azure or GCP Big Query and big data technologies like Spark, Flink etc.
• Solid grasp of Kubernetes, CI/CD and Terraform
• Working knowledge of ETL and ELT frameworks and orchestration tools like Airflow and dbt
• Ability to provide clear input, guide, empower junior members of the team to achieve desired outcomes
• Familiarity with machine learning and AI tools is desirable
• Knowledge of cybersecurity principles and practices