AI QA Test Engineer

POSITION SUMMARY

We are seeking a skilled QA Engineer with 4+ years of experience to drive end-to-end quality assurance for AI-powered applications. The ideal candidate will have strong foundations in both manual and automation testing, along with hands-on exposure to evaluating LLM-based systems. This role will own the quality lifecycle for AI-driven features, ensuring accuracy, reliability, safety, and performance across releases.

KEY RESPONSIBILITIES:

Manual Testing

• Create and execute test plans, test cases, and scenarios.

• Perform functional, UI, integration, regression, and exploratory testing for AI and non-AI modules.
• Validate end-to-end user journeys.

• Log defects with clear reproduction steps and collaborate with engineering teams for triage and resolution. Automation Testing

• Develop and maintain automated test suites.
• Build and enhance automation frameworks using Selenium, Playwright, or PyTest.

• Automate test execution for AI workflows where feasible.

• Integrate automated tests into CI/CD pipelines for continuous validation.

• Improve automation code quality, reliability, and coverage. AI / LLM Quality Assurance
• Test GenAI applications powered by LLMs.

• Evaluate LLM outputs for correctness, factual accuracy, and hallucinations.

• Develop structured evaluation datasets and golden test cases.

• Perform regression testing for prompt changes, dataset updates, and model upgrades. Quality Governance
• Ensure adherence to QA processes and documentation standards.

• Participate in sprint planning, requirement discussions, and design reviews.

• Support root-cause analysis for production defects and contribute to preventive actions.

REQUIRED SKILLS AND QUALIFICATIONS:

• 2–7 years of hands-on QA experience

(manual and automation).

• Strong understanding of QA methodologies, SDLC, and defect lifecycle.

• Experience testing AI/ML applications.

• Hands-on experience with test automation tools

• Experience with Git and Agile methodologies.

• Strong analytical, communication, and problem-solving skills.

EDUCATION:

• Bachelor’s degree in computer science, Engineering, or related field.