More jobs:
AI Evaluation Engineer
Job in
London, Greater London, W1B, England, UK
Listed on 2026-08-20
Listing for:
TXP
Full Time
position Listed on 2026-08-20
Job specializations:
-
Software Development
AI Engineer (Applied/Software), AI QA / Validation Engineer, AI Reliability/ Performance Engineer, Machine Learning/ ML Engineer
Job Description & How to Apply Below
Rate / Duration / Location
£700-£750/day Inside IR35 | Duration: 4 months | Clearance: BPSS + SC eligible |
Location:
London, Bristol or Manchester | Hybrid: 2 days onsite per week.
Overview
We're looking for an experienced AI Evaluation Engineer to join a specialist Public Sector Technology team. This hands-on role focuses on building evaluation frameworks, tooling and harnesses for AI systems, particularly LLM and agentic AI solutions. The role requires ownership, experimentation and rapid delivery.
Key Responsibilities
Design evaluation frameworks and tooling
Develop evaluation harnesses
Evaluate model and agentic AI systems
Define metrics and methodologies
Build repeatable evaluation processes
Investigate AI failures
Work directly with government teams
Prototype and test solutions
Present findings and recommendations
Contribute to strategic direction.
What We Are Looking For
Strong software engineering experience
Experience building and evaluating AI systems
Understanding of LLMs and agentic AI
Experience with AI evaluation frameworks
Ability to code independently
Strong problem-solving and communication skills
Comfortable working autonomously.
Technical Experience
AI/LLM Evaluation, RAG Evaluation, Ragas, Agentic AI Evaluation, Evaluation Harnesses, LLM Testing and Benchmarking, Prompt Engineering, Python Development, Model and Agent Integration.
Ideal Background
AI Evaluation Engineer, Applied AI Engineer, AI Engineer, LLM Engineer, Harness Engineer, Prompt Engineer or Software Engineer specialising in AI.
Team
Small Public Sector Technology team of approximately 2 to 6 people within a wider organisation.
What Will Make Someone Stand Out
Experience building evaluation tooling, evaluating LLMs, using Ragas, identifying AI failure modes, combining engineering with strategic thinking, and thriving in fast-moving environments.
Summary
Hands-on AI Evaluation Engineer role supporting government AI initiatives through evaluation, testing, benchmarking and framework development
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×