Data Engineer – Databricks Specialist; SC Cleared

Position: Data Engineer – Databricks Specialist (SC Cleared)
Location: Farringdon

Data Engineer – Databricks Specialist (SC Cleared)
SR2 is supporting a high-impact greenfield transformation programme within the public sector. We are looking for a hands-on Data Engineer with deep experience in Databricks and modern data architectures to help shape and build the foundation of a new cloud-native data platform from the ground up.

This role offers the opportunity to directly influence data engineering best practice, advise on architectural decisions, and collaborate with stakeholders across government, hyperscalers, and delivery teams to ensure performance, scalability, and compliance from day one.

Key Responsibilities:

Design and implement streaming and batch data pipelines in Databricks to support complex public sector use cases.
Lead the setup of a new Databricks environment, including security, data governance, orchestration, and performance tuning.
Collaborate with hyperscalers (e.g. AWS) to conduct Proofs of Concept (PoCs) for new data services and platforms.
Document architecture, performance benchmarks, and technical recommendations to support future scaling and support.

Required Experience &

Skills:

Proven experience as a Data Engineer delivering solutions in Databricks (across both streaming and batch).
Strong hands-on knowledge of cloud platforms, particularly AWS
Experience in building data solutions from scratch, including environment setup, configuration, and integration.
Programming proficiency in Python and/or Scala for data transformation, orchestration, and automation.
Demonstrable experience in conducting PoCs and documenting technical outcomes