Senior Spark Developer

Job Description
* ** Please wait...
** Senior Spark Developer Req

Start Date:

23/04/2026

Job Function:
Software Engineering Division: UK Business

Job Location:

G  Cheltenham - Project Offic, G  Ipswich - Adastral Hub Advertised Salary:
Competitive salary with benefits
** Recruiter:
Lloyd Hughes
**** Career Grade: D
**** Internal Closing Date: 1st May 2026
** Due to the sensitive nature of this role, you may be required to undergo DV (Developed Vetting) level Security Clearance (. An allowance of £5k per annum may be payable monthly while you hold this DV and continue to work in a role that requires that level of security clearance. The terms of this allowance will be made available if you are successful in being recruited into this role.

Security isn’t always the first thing that comes to mind when you think of BT, but when it comes to keeping everyone safely connected, We Are The Protectors. We deal with thousands of cyber-attacks every day, so that millions of people can safely go about their daily lives and run their businesses.
We deliver vital work at scale, with real breadth and impact. We connect for good.  You'll be joining a specialist security team that is a trusted partner to governments worldwide, protecting critical national infrastructure and committed to the safety and security of our nation and global communities. Our mission focused work is innovative, inspiring and technologically challenging in a way that makes every day different and stimulating.

We provide the opportunity to work on rare projects, with exciting tools and brilliant people. Everyone has access to unparalleled professional and personal development opportunities and your contribution is always valued.##
** About this role
*** This role sits within the Cyber Processing & Data Science engineering team, supporting the development of data processing capabilities that underpin critical cyber security services. You will contribute to the design and delivery of high quality Spark based solutions that are reliable, scalable and maintainable.
* As a Senior Spark Developer, you will be a hands-on engineer with strong knowledge of Apache Spark and its surrounding ecosystem. You will demonstrate technical leadership through the quality of your work, applying engineering best practices, improving how solutions are built and operated, and supporting others through collaboration and knowledge sharing. The role values curiosity, innovation and a clear focus on delivering reliable outcomes for customers.
** This role requires 5 days a week onsite. Mon - Fri 37.5 hours per week. DV Security Clearance is required. Must have lived in the UK for 10+ years to obtain clearance.**##
** What you’ll be doing
*** Build high quality Spark solutions that customers can rely on
* Design, develop and maintain well tested Spark applications and data pipelines. Ensure solutions are robust, performant and maintainable by applying agreed engineering standards and best practices.
* Support continuous improvement through technical leadership
* Lead development tasks by contributing to design discussions, reviewing code and sharing practical knowledge. Challenge inefficient ways of working, introduce improvements where appropriate, and support other engineers by demonstrating good practice and explaining technical concepts clearly.##
** Essential Experience
*** Strong hands-on experience developing Apache Spark applications using Scala
* Good working knowledge of the Spark ecosystem, including Spark SQL and Data Frames
* Experience building and supporting data pipelines and ETL processes
* Solid understanding of software engineering best practice, including testing, code quality and version control
* Experience working with containerised platforms such as Docker and Kubernetes or Open Shift##
** Desirable Experience
*** Experience with Kafka for data ingestion
* Experience with workflow or orchestration tools such as Airflow or Ni Fi
* Familiarity with cloud object storage (e.g. Azure Blob Storage or S3)
* Experience using monitoring and observability tools such as Prometheus, Grafana or Spark UI
* Working knowledge of Linux environments and basic scripting (e.g. Bash or…