Skip to content
×

Senior QA & Automation Engineering Leader – Conversation AI Assurance and Test Automation

Listing for: Resillion
Part Time position Listed on 2026-11-12
Job specializations:
Salary/Wage Range or Industry Benchmark: 90000 - 130000 GBP Yearly GBP 90000.00 130000.00 YEAR

Job Description & How to Apply Below

Senior QA & Automation Engineering Leader – Conversation AI Assurance and Test Automation

Location: Central Scotland

Working Arrangement: Hybrid, with travel to Glasgow (2 Days per week)

Job Type: Permanent or Contract (Inside IR35)


** Due to background-check requirements, applicants must have a minimum of three years' UK address history. **

About Resillion

Resillion is a global provider of digital assurance, cybersecurity, and testing services. We help organisations strengthen their security posture, protect critical assets, and deliver high-quality digital solutions. Our teams operate across multiple specialist disciplines, including Digital Forensics, Incident Response, SOC Operations, Quality & Compliance, and Technical Consultancy. We are committed to excellence, continuous improvement, and delivering trusted, customer-focused services that meet the highest professional and accreditation standards.

Role Overview

We are seeking an experienced Senior QA & Automation Engineering Leader to take ownership of AI testing, quality assurance and automation across enterprise-scale technology programmes.

The successful candidate will combine strategic QA leadership with hands‑on technical expertise, leading the testing and assurance of Conversational AI, Generative AI, Large Language Models (LLMs), Retrieval‑Augmented Generation (RAG) and agentic AI solutions.

Proven experience testing Conversational AI systems (Chat and/or Voice) is essential. Candidates must demonstrate practical experience owning AI testing, leading teams, managing delivery and driving measurable improvements in conversational AI quality.

The role will involve working closely with AI Engineers, Data Scientists, Software Engineering teams and business stakeholders to ensure AI solutions are accurate, reliable, secure and fit for production.

Key Responsibilities Conversational AI and AI Assurance
  • Own and drive the end-to-end testing strategy for Conversational AI solutions across chat and voice platforms.
  • Lead testing of LLM-powered applications, chatbots, voice assistants, RAG solutions and agentic AI workflows.
  • Validate conversational accuracy, contextual understanding, intent recognition, response consistency, multi‑turn interactions and dialogue management.
  • Test voice capabilities, including speech recognition (ASR), text‑to‑speech (TTS), response latency and interruption handling.
  • Assess AI‑generated responses for accuracy, relevance, hallucinations, bias and adherence to business requirements.
  • Validate prompt handling, retrieval accuracy, context management, integrations and AI guardrails.
  • Establish AI testing frameworks, evaluation benchmarks, regression suites and measurable quality metrics.
AI Testing Leadership and Delivery
  • Take ownership of AI testing strategy, execution, quality outcomes and delivery accountability.
  • Lead and develop QA and automation engineering teams, setting technical direction and delivery priorities.
  • Drive conversational AI quality, continuous improvement and production readiness.
  • Work closely with AI Engineering, Data Science, Data Engineering and business stakeholders.
  • Communicate testing progress, quality risks, defects and release recommendations to senior stakeholders.
Test Automation and Engineering
  • Architect and maintain scalable automation frameworks using Selenium, Playwright, Cypress, UFT, Python, Java and Cucumber BDD.
  • Develop automated UI, API, integration, mobile, backend and data validation test suites.
  • Integrate automated testing into Azure Dev Ops, Jenkins and Git Hub Actions CI/CD pipelines.
  • Drive automation coverage, regression efficiency and continuous testing.
  • Apply AI‑assisted testing approaches using Git Hub Copilot and Microsoft Copilot, with appropriate validation.
Data, API and Enterprise…

Position Requirements

10+ Years work experience
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary