Lead AI Test Engineer

Location: Greater London

Job Title :
Lead AI Test Engineer Location :
London About the Job you are considering:

We are seeking an experienced Lead AI Test Engineer to drive quality assurance and testing strategies for AIML powered applications Generative AI solutions and intelligent automation platforms The ideal candidate will combine strong expertise in software test engineering with handson experience validating AI models LLM based applications machine learning systems and data driven solutions across the full development lifecycle.

Hybrid working :

The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.

Your Role:

Define and implement end-to-end testing strategies for AI Machine Learning and Generative AI solutions

Lead QA activities across AI product development ensuring quality reliability performance security and compliance

Design and execute test plans for LLM powered applications chatbots copilots RAG solutions and AI agents

Validate AI model accuracy bias fairness explainability robustness and performance

Develop and maintain AI test automation frameworks using Python and modern testing tools

Perform functional integration regression API performance and security testing for AI platforms

Establish test data management practices and synthetic data generation approaches

Define and monitor AI quality metrics model drift hallucination rates response consistency and user experience KPIsCollaborate with Data Scientists AI Engineers Product Owners Architects and business stakeholders

Support CICD integration and quality gates for AI model deployment pipelines

Mentor QA teams and promote best practices in AI testing and automation

Your

Skills:

10 years of experience in Software Testing Test Automation and Quality Engineering Handson experience testing AIML and Generative AI solutions

Strong knowledge of Large Language Models LLMs Retrieval Augmented Generation RAG Prompt Engineering and AI Agents

Experience with AI platforms such as Azure OpenAI Microsoft Copilot Studio OpenAI AWS Bedrock Vertex AI or similar

Proficiency in Python and test automation frameworks

Experience with API testing tools such as Postman Rest Assured or similar

Good understanding of Machine Learning model validation techniques and evaluation metrics

Experience with CICD pipelines Git Azure Dev Ops Jenkins or Git Hub Actions Knowledge of security privacy ethical AI and responsible AI principles

Familiarity with cloud platforms including Azure AWS or GCP.Desirable (Not Mandatory):

Experience implementing AI testing frameworks and governance controls

Knowledge of MLOps and model lifecycle management

Understanding of AI risk management and regulatory compliance frameworks

Certifications in AI Cloud Quality Engineering or Test Automation We are a Disability Confident

Employer:

Capgemini is proud to be a Disability Confident Employer (Level
2) under the UK Government’s Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:

Declare they have a disability, andMeet the minimum essential criteria for the role.

Please opt in during the application process.

Make It Real (what does it mean for you):

You’d be joining an accredited Great Place to work for Wellbeing in 2024. Employee wellbeing is vitally important to us as an organisation. We see a healthy and happy workforce a critical component for us to achieve our organisational ambitions.

To help support wellbeing we have trained ‘Mental Health Champions’ across each of our business areas, and we have invested in wellbeing apps such as Thrive and Peppy.

You will be…