Principal AI Quality Engineer

Position Overview

This is an exciting opportunity to define how a global technology company assures quality in the age of AI. Fourth is seeking an exceptional, strategically minded Principal AI Quality Engineer to drive our AI‑driven global QA strategy from the front. Reporting to the Global Head of Service Delivery within Technology, you will be the most senior quality voice in the organisation, shaping how quality is engineered across Fourth’s systems in an era of AI‑assisted and agentic AI development.

You will set the vision, standards, and frameworks that build quality, reliability, and trust into everything we ship, aligning the QA strategy directly with the company’s technical vision, roadmap, and business strategy. If you want to leave a lasting mark on how software quality and AI come together at scale, this is the role to do it.

• Define, own, and continuously evolve Fourth’s AI‑driven global QA strategy and roadmap, aligned with the Product and Technology roadmaps, to shape the strategic testing needs of the business.
• Establish the QA tools, frameworks, standards, and ways of working across our global engineering organisation to achieve our technology goals.
• Drive the unification and continuous improvement of testing processes and practices across delivery teams, partnering directly with Software Engineering Managers and Product Managers.
• Act as the principal point of escalation for complex testing and quality challenges across global Engineering teams, marshalling subject‑matter expertise to resolve them.
• Champion modern testing technologies, frameworks, and tools across Engineering, including the responsible adoption of AI.
• Define and own the quality KPIs and quality gates that measure product quality and the effectiveness of AI‑assisted delivery.
• Own the Engineering Defect Management process and the organisation‑wide approach to defect prevention.
• Lead Fourth’s QA approach for AI development end to end, covering both AI‑assisted development (such as AI code generation and developer copilots) and agentic AI development (autonomous and semi‑autonomous systems).
• Establish quality, evaluation and governance frameworks for non‑deterministic and AI‑driven systems, including test and evaluation strategies for LLM‑based and agentic features.
• Set enterprise standards for validating AI‑generated code, ensuring it meets Fourth’s quality, security, and maintainability requirements.
• Partner with Engineering, Product, and Service Delivery leadership to embed AI fluency and modern quality practice across the organisation, mentoring and uplifting QA capability globally.

• Strong understanding of the testing and quality challenges of non‑deterministic, ML/LLM‑based, and agentic systems, including evaluation, guardrails, and the regression of model‑driven behaviour.
• Proven experience defining and implementing QA strategy at an organisational or enterprise scale.
• Experience establishing test automation best practice, both functional and non‑functional, across multiple teams.
• Demonstrable experience shaping or leading a QA approach for AI‑assisted and/or agentic AI development.
• Track record of defining and analysing quality KPIs and quality gates.
• Ability to influence and provide technical leadership and mentoring across senior engineering and product stakeholders.
• Experience creating and owning technical strategy and process documentation.
• Strong experience with Agile software delivery at scale.
• Excellent communication and stakeholder‑management skills, with fluent English (written and spoken).

• Familiarity with MLOps or LLMOps practices, and with monitoring and observing the behaviour of AI systems in production.
• Exposure to prompt design and prompt/response evaluation techniques for AI‑enabled features.
• Strong programming proficiency in C# or Java.
• Expertise in web and API testing, including hands‑on UI automation with modern tools such as Playwright (or comparable frameworks).
• Proven ability to design and implement robust, scalable QA test strategies and automation frameworks.
• Hands‑on experience with performance, security and…