Data Engineer

Position: Data Engineer - Contract - 9+ Months
Location: City

Data Engineer (Contract)
9+ Month Contract based in Sheffield
395 - 442 per day InsideIR
35
BPSS clearance required - candidates must be eligible

My client is seeking a Data Engineer to design and operate large-scale telemetry and observability data pipelines within a modern Open Shift and Kafka ecosystem. This role is central to enabling proactive, Level 4 observability, delivering high-quality metrics, logs, and traces to support platform reliability, operational insight, and automation.

Responsibilities:

Design, implement and maintain scalable data pipelines to ingest and process
Open Shift telemetry (metrics, logs, traces)
Stream telemetry through Kafka (producers, topics, schemas) and build resilient
consumer services for enrichment and transformation
Engineer multi-tenant observability data models, ensuring data lineage, quality
controls and SLAs across streaming layers
Integrate processed telemetry into Splunk for dashboards, analytics, alerting and
operational insights
Implement schema management and governance using Avro/Protobuf, including versioning
and compatibility strategies
Build automated validation, replay and backfill mechanisms to ensure data
reliability and recovery
Instrument services using Open Telemetry, standardising tracing, metrics and
structured logging
Apply LLMs to enhance observability, such as query assistance, anomaly summarisation
and runbook generation
Collaborate with Platform, SRE and Application teams to align telemetry, alerts and
SLOs
Ensure pipelines meet security, compliance and best-practice standards
Produce clear documentation covering data flows, schemas, dashboards and operational
runbooks

Skills & Experience:

Strong hands-on experience building streaming data pipelines with Kafka
(producers/consumers, schema registry, Kafka Connect, KSQL/KStreams)

Experience with Open Shift / Kubernetes telemetry, including Open Telemetry and
Prometheus
Proven capability integrating telemetry into Splunk
(HEC, Universal Forwarders, source types, CIM, dashboards, alerting)
Solid data engineering skills in Python (or similar) for ETL/ELT, enrichment and
validation

Please apply for immediate interview!

CBSbutler is operating and advertising as an Employment Agency for permanent positions and as an Employment Business for interim / contract / temporary positions. CBSbutler is an Equal Opportunities employer and we encourage applicants from all backgrounds.