Ai Test Lead; Ai Foundry

We are looking for an experienced AI Test Leads (AI Foundry) to ensure AI and Copilot solutions are safe, reliable and compliant, covering both traditional QA and AI specific risks (bias, hallucination, explainability).

The role defines assurance methods, quality gates and postdeployment monitoring to meet internal policy and regulator expectations.

• Design and manage the enterprise testing strategy for AI/Copilot, blending traditional QA with AI specific methods.
• Define test approaches for functional, performance, accuracy, reliability, ethical compliance and bias detection.
• Establish model evaluation techniques (prompt variability, edgecase simulation, output consistency, scenario reasoning).
• Validate explainability, traceability and safety controls against policy and regulatory requirements.
• Evaluate and test humanintheloop workflows and decision checkpoints for appropriate oversight.
• Embed quality gates in iterative delivery, preventing progression without assurance evidence.
• Develop and maintain specialised test datasets, including adversarial, low quality, domain specific and edgecase inputs, to rigorously challenge model robustness and identify systemic weaknesses.
• Provide AI test engineering support to delivery squads, advising on model readiness criteria, testability risks, and quality implications of design decisions, ensuring solutions are verifiable throughout the lifecycle.
• Define and run postdeployment validation, drift detection, incident triage and continuous model monitoring.
• Partner with Risk, Legal, Security and Compliance teams to meet control frameworks and audit standards.
• Provide inputs to risk/impact assessments, policy adherence checks and governance submissions.
• Lead incident investigations for unexpected AI behaviours, conducting deepdive rootcause analysis across data quality, model logic, prompt flows, integration layers and humanintheloop steps; identify systemic failure points, recommend corrective actions, and drive endtoend remediation to prevent recurrence.
• Maintain test documentation, evaluation logs, datasets and reproducible evidence for audit.
• Uplift AI testing capability across teams through standards, templates, training and handson support.
• Champion continuous improvement of AI assurance, evaluating new testing tooling (LLMmonitoring, bias scanners, prompt diff tools, synthetic data generators) and maturing standards as organisational AI adoption scales.
• Ensure responsible AI principles (e.g., transparency, explainability, ISO
  42001) are incorporated into all development.
• Provide insight to support business cases, investment decisions, risk assessments, and prioritisation discussions at AI governance forums.
• Managing escalations supporting the wider Data & AI Leadership team.

• Higher education qualification (or equivalent experience) in Ethics, Law, Risk Management, Social Sciences, Data/Computer Science or relevant field
• Experience with designing and leading testing for complex digital or data driven systems, including multicomponent architectures, API integrated platforms, event driven workflows and systems operating under regulatory or high assurance constraints.
• Clear understanding of AI specific risks such as hallucinations, bias, drift, explainability gaps, safety breaches and misuse pathways, paired with the ability to design targeted tests that uncover model blind spots and systemic weaknesses.
• Knowledge of model evaluation techniques, prompt testing strategies and scenario based testing approaches, including stress testing prompts, adversarial input creation, failure mode exploration and behaviour driven evaluation.
• Familiarity with governance, audit and regulatory standards for AI, data and digital services, ensuring testing evidence aligns with internal risk frameworks, ISO
  42001 controls, Responsible AI policies and external regulatory expectations.
• Experience developing structured QA strategies that integrate traditional and AI specific assurance, mapping out test plans, riskbased prioritisation, acceptance criteria, model readiness thresholds and quality gates aligned to lifecycle stages.
• Ability to define and execute test plans across functional, nonfunctional, ethical and performance dimensions, validating accuracy, latency, robustness, security, fairness, reliability and userjourney consistency.
• Strong analytical mindset with the ability to identify root causes of defects or unexpected AI…