Lead AI Quality Engineer

Our remote first employees join and stay at Phase2 to be a force. In pursuit of digital experience that advances the human experience, you will do your best work, live fully, and succeed as a team.

Phase2 a digital agency passionate about customer experience, grounded in data & insights, and rooted in more than two decades of successful technology delivery. We guide industry-leading companies to what’s next in their digital landscape.

And it all begins with finding the right people who align with our values and passion for impact.

The Lead AI Quality Engineer owns the evolution of quality assurance king closely with leaders within the Engineering Department, including other Quality Specialists, you will lead setting the vision for how Phase2 maintains a high level of quality in the software we produce, in an era when our engineers’ use of agentic AI has led to unprecedented rates of code production.

You will also own setting and executing a roadmap that implements this vision: establishing consistent guidelines, methodologies, and tools that serve as the standard for quality assurance across our projects, and building a quality system capable of keeping pace with AI-accelerated delivery.

Your ownership of quality assurance spans two fronts: testing our AI-powered products and features, and applying AI-driven tools and techniques to test our legacy software systems. This includes establishing standards and guidelines for creating test plans for AI-powered features (LLM-based applications, agentic workflows, ML-driven components), and for using AI-powered testing tools responsibly with appropriate human oversight. Your ownership of quality strategy runs end-to-end, from requirements and design through production monitoring, across complex digital and AI-powered delivery engagements.

This is a hybrid role: allocated half-time to work on client projects, and half-time to overseeing and growing the Quality practice. Your direct work with clients involves leading quality strategy design and execution across one or more client delivery teams, presenting quality outcomes directly to client stakeholders, and coaching Phase2 project team members and client teams toward a quality ownership mindset.

Your Quality practice oversight responsibilities include setting the vision and roadmap, executing on that plan, and managing, mentoring, and coaching other members of the quality team.

In collaboration with the VPs of Engineering, you will help socialize our approach to quality across other departments, including project management, strategy, sales, and marketing, ensuring that everyone has a shared understanding of the vision and how it is being executed.

This position reports to: VP, Engineering

• Quality strategist and technical leader: You've built your career hands-on in QA, and you're ready to own quality strategy rather than just execute test plans. You can translate coverage decisions into a business narrative and advise clients directly.
• Systems thinker for quality: You design coverage models anchored to business-critical outcomes, and you're hands-on, building automation frameworks and evaluation pipelines yourself.
• AI-native testing mindset: You bring a modern, AI-first testing approach to every project, including directing AI-powered testing tools, using prompt engineering to generate adversarial test scenarios, and evaluating LLM and ML outputs for accuracy, hallucination, and bias.
• Technical range and adaptability: You are resourceful across unfamiliar client stacks and delivery models, ramping up quickly on new tools and AI systems through hands-on exploration and collaboration with subject-matter experts.
• Client-facing consultant: You act as a quality consultant to…