Secure Data Engineer

Who You Will Be Working With...

The Data Platforms team is part of the Insights and Data Global Practice and has seen strong growth and continued success across a variety of projects and sectors. Data Platforms is the home of the Data Engineers, Platform Engineers, Solutions Architects and Business Analysts who are focused on driving our customers digital and data transformation journey using the modern cloud platforms.

We specialise on using the latest frameworks, reference architectures and technologies using AWS, Azure and GCP along with various data platforms like Databricks, Snowflake, Quantexa, Palantir, SAS.

Security Clearance: To be successfully appointed to this role, must be eligible to obtain Security Check (SC) clearance.
To obtain SC clearance, the successful applicant must have resided continuously within the United Kingdom for the last 5 years, along with other criteria and requirements.

Throughout the recruitment process, you will be asked questions about your security clearance eligibility such as, but not limited to, country of residence and nationality.
Some posts are restricted to sole UK Nationals for security reasons; therefore, you may be asked about your citizenship in the application process.

The Focus Of Your Role

We are looking for a code first Data Engineer to design and build scalable and resilient data applications for Defence customers. This role sits at the intersection of Software Engineering and Data Engineering, working on mission critical systems where reliability, security and auditability are essential.

You will build data applications that process high volume and high velocity data, orchestrate complex workflows, and deploy your own solutions into secure containerised environments. You will work across the full lifecycle from development through to production, shaping architectures that support operational users, analytics, and AI enabled capabilities.

This role is hands on and engineering. You will write production grade code, contribute to secure platform patterns, and ensure that your data services run predictably in tightly governed Defence environments.

What You Will Bring

• Core Engineering
  Expert level Python and strong foundations in software engineering such as object oriented design, automated testing and version control.

• Data Engineering Stack
  Experience building pipelines using streaming frameworks, distributed processing engines and relational or analytical storage technologies.
  Examples include Kafka, Spark, Postgre
  
  SQL.

• Orchestration
  Experience defining and running data workflows using modern orchestration frameworks.
  Examples include Airflow, Dagster or Prefect.

• Data Quality and Lineage
  Familiarity with tools and techniques for data testing, documentation and lineage.
  Examples include Great Expectations or dbt.

• AI and MLOps
  Understanding of how to operationalise machine learning models in production, including model packaging, monitoring and controlled deployment.

• Containerisation and Kubernetes
  Confidence deploying applications in containerised environments, including defining services, pods and deployment configurations.

Examples include Docker and Kubernetes.

• Dev Ops Mindset
  Hands on experience with CI/CD approaches and a belief in owning the services you build.
  Examples include Git Lab CI, Git Hub Actions or Argo.

• Nice to have
  
  Experience with Infrastructure as Code or configuration management tools.
  Examples include Terraform or Ansible.
  Experience working in secure, restricted or air gapped environments, including Defence networks or MODCloud aligned platforms.
  Familiarity with Google Distributed Cloud (GDC) or other edge and on premises cloud platforms used in constrained or disconnected settings.

What You Will Be Doing

Your work will involve

Building Data Applications
Developing modular and maintainable software components that process, transform and expose data for analytical, operational or AI driven use cases. You will follow strong engineering practices, with testing and observability built in from the start.

Streaming and Real Time Architecture
Designing and implementing…