Senior AI Evaluation Engineer

Senior AI Evaluation Engineer (Data Scientist) - London/Manchester/Bristol

The Incubator for AI (i.AI) is a team of around 80 builders and operators using frontier AI to improve public services across healthcare, education, planning and parliament. We work at startup pace, with the backing of the Prime Minister and No10, on problems that affect millions of people.

Our Impact and Evaluation team embeds within product teams to build rigorous evaluation into every stage of development, shaping product decisions and showing whether our products improve public services. We’re now hiring two Senior AI Evaluation Engineers to focus on model-level evaluation of accuracy, bias and safety.

• Leading the design of evaluation pipelines and architecting product-wide evaluation suites that integrate with engineering and CI/CD workflows
• Developing complex and exploratory methods to assess accuracy, bias, safety and model performance
• Working with product, engineering and domain experts to shape evaluation across the product lifecycle and inform technical decisions
• Turning complex results into clear recommendations and leading communication of findings through reports, presentations and external publications
• Reviewing and supporting other engineers’ work to raise standards and build evaluation capability across the team

You don’t need to tick every box. You’ll likely bring some of the following:

• Broad experience evaluating generative and predictive AI systems, including LLM-as-a-judge and tools such as pydantic evals, deepeval or inspect
• Extensive knowledge of experimental design, statistical hypothesis testing and quantitative research methods (particularly those in the Magenta Book)
• High proficiency in Python and strong software practices, including git, CI/CD, unit testing and pipeline orchestration tools like DVC or snakemake
• The ability to communicate technical work to broad audiences through reports, presentations and web-based tools
• Experience mentoring junior colleagues

• Career-defining work. Projects backed by No10 that apply AI to public services affecting millions, alongside talented, mission-driven colleagues
• The tools to ship. Frontier models, ample compute, access to experts in national security, policy and AI research, and full operational and delivery support
• Early ownership of important products in small teams, 5 L&D days a year, and funding for learning, conferences and collaborations
• £63,165–£86,640 in London (£58,275–£81,885 nationally), including a technical allowance of up to £15,600, plus a 28.97% employer pension contribution
• Flexibility and benefits. Hybrid working from London, Manchester or Bristol, 25+ days’ leave, up to 39 weeks’ full-pay parental leave, and perks including cycle to work and dental insurance