Intermediate Data Engineer

Job Description

Build, maintain, and improve batch and near‑real‑time data pipelines that support analytics and operational use cases.

Contribute to ingestion, transformation, storage, and serving layers within the enterprise data platform.

Develop reliable and maintainable data processing logic using SQL, Python, and related tooling.

Help modernise legacy data workflows into more scalable and supportable patterns.

• Build, maintain, and improve batch and near‑real‑time data pipelines that support analytics and operational use cases.
• Contribute to ingestion, transformation, storage, and serving layers within the enterprise data platform.
• Develop reliable and maintainable data processing logic using SQL, Python, and related tooling.
• Help modernise legacy data workflows into more scalable and supportable patterns.
• Contribute to the implementation and improvement of data lake or lakehouse components.
• Support the preparation of data for efficient analytical use across warehouse and large‑scale storage environments.
• Assist with data layout, partitioning, and optimisation practices that improve platform performance and maintainability.
• Work with senior engineers to apply platform standards and architectural patterns consistently.
• Build and maintain data quality checks, reconciliation logic, and pipeline validation routines.
• Help ensure datasets meet expectations for freshness, completeness, and accuracy.
• Support metadata, lineage, and documentation practices that improve trust and transparency.
• Troubleshoot pipeline failures, data discrepancies, and performance issues in production.
• Partner with BI, analytics, software engineering, product, and business stakeholders to understand and support data use cases.
• Translate business and platform requirements into scalable, well‑engineered technical solutions.
• Participate in planning, estimation, implementation, testing, and deployment activities.
• Contribute to code reviews, documentation, and shared engineering standards.
• Identify opportunities to improve automation, reliability, scalability, and maintainability across the platform.
• Stay current with emerging data engineering practices, tools, and patterns.
• Grow technical capability in modern data platform engineering through hands‑on project delivery.

The platform may include a combination of established and modern technologies such as SQL Server, Python, Spark, Flink, Airflow, Object storage, Open format Tables, Kafka or Redpanda, Click House or similar columnar analytical stores, Git and CI/CD tooling, Kubernetes, Open Shift, or similar runtime environments.

• Degree or diploma in IT, Computer Science, Engineering, or a related technical discipline.
• 2 to 4 years of experience in data engineering, ETL/ELT development, or data platform engineering.
• Hands‑on SQL experience, including query optimisation, indexing, and performance tuning.
• Experience building and maintaining batch data pipelines in production environments.
• Exposure to data platform concepts such as data warehouse, data lake architecture, or object storage.
• Experience with orchestration platforms such as Airflow, Dagster, or similar.

• Engages in cross‑functional collaboration and problem solving while contributing to an inclusive team culture.
• Supports a culture of adaptability and shared accountability across the department and wider business.
• Shows up authentically and contributes to team success by working effectively with diverse colleagues and perspectives.
• Approaches challenges as opportunities to learn, improve, and help others grow.