Data Engineer – Datacore, IOTT

Responsibilities

• Design robust, secure, and supportable data solutions for both batch and streaming use cases, ensuring scalability, resilience, and security by design
• Develop, test, and support data pipelines integrating data from industrial, IoT, and enterprise systems using technologies such as Azure Synapse, and event streaming platforms (e.g. Event Hubs, Kafka, MQTT, Quix)
• Work with edge and integration platforms (e.g. High Byte, Ignition, Quix, or similar) to ingest, model, and standardise operational data, aligned to a Unified Namespace (UNS) approach for organising and distributing real-time operational data
• Build and optimise solutions for handling time-series and telemetry data, ensuring performance and usability at scale
• Collaborate with architects, cloud engineers, and platform teams to design and evolve data platform capabilities, including CI/CD, deployment automation, and environment management
• Ensure data quality, observability, and reliability across real-time and batch pipelines
• Where required, contribute to downstream data modelling and analytics to support reporting and insight
• Build automated test frameworks and develop test harnesses for each component, increasing overall test coverage and ensuring a culture of quality and reliability across the platform.

• Strong experience in data engineering using cloud platforms (e.g. Azure), including tools such as Azure Synapse, Spark, Python, and SQL
• Experience working with real-time or streaming data pipelines (e.g. Event Hubs, Kafka, MQTT, Quix, or similar)
• Experience working with time-series or telemetry data
• Experience integrating data from APIs, industrial systems, or edge platforms (e.g. High Byte, Ignition, or similar)
• Familiarity with Unified Namespace (UNS) principles and MQTT-based architecture is advantageous
• Experience in data modelling (e.g. relational modelling, dimensional modelling, or similar approaches suited to operational systems)
• Experience delivering end-to-end data solutions from ingestion through to production deployment
• Experience working in an Agile/Scrum environment
• Experience collaborating with platform and engineering teams to support CI/CD, Dev Ops, and engineering best practices
• Knowledge of Azure Dev Ops or similar tools
• Awareness of security, data governance, and working within regulated or operationally critical environments is desirable.