Quality Engineering Specialist

Location: Bengaluru

Quality Engineer Specialist – AI & LLM Solutions

We are looking for a  QA Engineer Specialist specializing in AI and LLM-based systems  to ensure the quality, reliability, and performance of AI products used across global media and marketing platforms. The role focuses on testing  LLMs, RAG pipelines, AI agents, and ML/data workflows , with a strong emphasis on  automation and AI-specific quality metrics .

Key Responsibilities
Own quality assurance for  AI products , including LLMs, RAG systems, agents, and ML pipelines.
Design and execute  AI-specific test strategies  (factuality, hallucinations, robustness, safety).
Validate  agent behaviors , AI outputs, and performance (latency, stability, quality).
Build and maintain  automated test suites  integrated into CI/CD pipelines.
Prevent regressions and assess the impact of  model, embedding, or prompt changes .
Collaborate with engineers and Product teams to challenge  acceptance criteria and system robustness .

Required Skills & Experience
7-9 years of experience in  QA or Test Engineering  (exposure to AI/ML systems preferred).
Strong experience with  test automation in Python .
Hands-on experience with  API testing  and  end‑to‑end testing frameworks .
Working understanding of  LLMs and RAG concepts  (embeddings, vector search).
Experience integrating tests into  CI/CD pipelines .
Basic experience with  Docker  and test environments.

Nice to Have

Experience with  AI/LLM evaluation techniques  (factuality, toxicity, consistency).
Exposure to  Azure OpenAI or Vertex AI  platforms.
Performance or load testing experience.

Why Join us
Play a key role in defining  AI quality and trust  for enterprise GenAI solutions.
Work on  production AI platforms  used by global media brands.
Be part of a growing  Media Tech Global Capability Center in India .