Lead Consultant - QA; AI) Engineer

At Delphi, we believe in creating an environment where our people thrive. Our hybrid work model empowers you to choose where you work - whether it's from the office, your home, or a mix of both - so you can prioritize what matters most. We are committed to supporting your personal goals, family, and overall well-being while driving transformative results for our clients.

We welcome exceptional talent from anywhere across the globe. Interviews and onboarding are conducted virtually, reflecting our digital-first mindset.

Rooted in the region, we specialize in delivering tailored, impactful solutions in Data, Advanced Analytics and AI, Infrastructure, Cloud Security, and Application Modernization
. Whether it’s enabling predictive analytics
, transforming operations with automation, or driving customer engagement with intelligent platforms, we are the trusted partner for organizations ready to embrace a smarter, more efficient future.

We are looking for a hands‑on Senior QA Consultant to lead end‑to‑end testing of AI and generative AI applications across enterprise environments. This role combines strong expertise in traditional QA engineering with modern AI evaluation and validation practices. The ideal candidate will drive quality assurance initiatives for RAG pipelines, multi‑agent systems, OCR and speech‑to‑text solutions while ensuring production‑grade quality outcomes in regulated industries such as finance, healthcare, and insurance.

The successful candidate will lead a small QA pod, collaborate closely with engineering, AI/ML, product, and business teams, and act as the client‑facing QA owner for enterprise AI engagements. This role requires strong technical leadership, automation expertise, AI evaluation capabilities, and excellent stakeholder management skills.

• 10–15 years of experience in Software QA / Test Engineering
• 3–4 years of experience leading QA teams or delivery pods
• Hands‑on experience testing AI / GenAI applications in production environments
• Experience working within regulated domains such as finance, healthcare, insurance, or similar industries

• Own end‑to‑end quality assurance for AI/GenAI engagements including test strategy, planning, execution, defect management, reporting, and exit criteria
• Lead and mentor QA pods of 3–6 engineers across automation frameworks, AI testing practices, and delivery execution
• Design and execute functional, API, backend, integration, regression, performance, and AI evaluation test cases
• Build and maintain scalable automation frameworks for web, mobile, and API testing
• Drive continuous quality improvements through reusable accelerators, frameworks, datasets, and evaluation tools

• Define AI‑specific test coverage including prompt validation, hallucination detection, groundedness checks, retrieval quality, tool‑call correctness, latency, bias, and safety testing
• Design and execute testing strategies for RAG architectures, multi‑agent workflows, OCR systems, and speech‑to‑text applications
• Build synthetic datasets covering edge cases, adversarial scenarios, persona‑based workflows, and multimodal inputs
• Implement LLM‑as‑a‑Judge evaluation pipelines using customizable scoring rubrics and multi‑model evaluations
• Produce detailed AI quality scorecards and benchmarking reports across accuracy, relevancy, faithfulness, and safety metrics

• Integrate AI evaluation and automation testing into Jenkins and Azure Dev Ops CI/CD pipelines
• Build regression dashboards, quality gates, and automated reporting frameworks
• Perform API testing, backend validation, database testing, and…