Senior Data Engineer - DV Cleared

Senior Data Engineer - Elastic Stack / Apache Ni Fi

📍 Worcester (Onsite)
💰 £70-£75 per hour | Outside IR35
📅 12-Month Rolling Contract
⏰ Compressed Working Available - Work 3 Long Days, Paid for 5

We are seeking an experienced Senior Data Engineer to join a high-profile programme delivering secure, scalable data platforms within a highly regulated environment. This is an exciting opportunity to work on complex data engineering challenges using cutting-edge technologies, with the flexibility of compressed working patterns.

The Opportunity

As a Senior Data Engineer, you will play a key role in designing, building, and maintaining robust data pipelines and platforms. You will work closely with architects, engineers, and security teams to deliver high-performance data solutions that meet stringent security and governance requirements.

This role is fully onsite in Worcester, with the option to work a compressed schedule (e.g. three extended working days while being compensated for a full five-day week).

Key Responsibilities

Design, develop, and maintain secure, scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache Ni Fi
Build and manage data ingestion, transformation, and integration processes
Ensure data quality, integrity, security, and compliance with governance standards
Collaborate with data architects and security teams within highly secure environments
Monitor and manage large-scale, real-time data flows, optimising performance and reliability
Develop robust data models to support analytics and reporting capabilities
Create dashboards and visualisations using Kibana to drive operational insights
Troubleshoot, debug, and tune data pipelines and platform performance
Implement high availability, backup, and disaster recovery strategies
Produce clear technical documentation, including architecture, workflows, and security protocols

Essential Skills & Experience

Strong experience with the Elastic Stack (Elasticsearch, Logstash, Kibana)
Proven expertise with Apache Ni Fi
Extensive experience designing and building enterprise-scale data pipelines
Strong understanding of secure data architectures and data governance
Experience operating within regulated or security-conscious environments
Ability to design and deliver end-to-end data engineering solutions
Excellent troubleshooting and performance optimisation skills

Desirable Experience

Experience working within government, defence, or other highly regulated sectors
Knowledge of Apache Kafka, Spark, or Hadoop

Experience with Docker and Kubernetes
Familiarity with monitoring and observability tools such as Prometheus, Grafana, or ELK
Understanding of machine learning algorithms and their implementation

Experience with Infrastructure as Code tools such as Terraform and Ansible

What's on Offer?

✅ £70-£75 per hour
✅ Outside IR35
✅ 12-month rolling contract
✅ Compressed working available (3 long days, paid for
5)
✅ Opportunity to work on complex, mission-critical data platforms
✅ Long-term contract stability within a high-profile programme

If you're an experienced Data Engineer with strong Elastic Stack and Apache NiFi expertise and are looking for a long-term, high-impact contract opportunity, we'd like to hear from you