AI QA Automation Engineer; Python
Job in
Boise, Ada County, Idaho, 83701, USA
Listed on 2026-09-26
Listing for:
Luxoft Germany
Full Time
position Listed on 2026-09-26
Job specializations:
-
Software Development
AI QA / Validation Engineer, Software Testing, AI Reliability/ Performance Engineer
Job Description & How to Apply Below
Project description
Own AI quality, automated testing, reliability engineering, and agent-powered QA automation for the platoon.
Responsibilities- Build automated evaluation frameworks for agents, RAG, prompts, tools, workflows, and model outputs.
Create benchmark datasets, golden responses, regression suites, evaluation metrics, and release thresholds.
Test hallucination, grounding, citation accuracy, prompt injection, tool selection, and multi‑agent coordination.
Develop agents that generate test cases, validate API and workflow outputs, and support intelligent UAT.Build UI, API, integration, performance, end‑to‑end automation, and CI/CD quality gates.
Monitor production quality, drift, reliability, and recurring failure patterns.
Must have
- Bachelor's degree in Computer Science, Software Engineering, Information Systems, or a related field.
5+ years of software quality engineering, test automation, or reliability engineering experience.
Strong Python skills and experience building reusable automated test frameworks.
Experience with API, UI, integration, regression, and CI/CD‑based testing, including measurable quality gates.
Ability to translate requirements, business workflows, and AI behaviors into measurable quality criteria.
Experience testing generative AI, RAG, copilots, or AI agents.
Experience with Playwright, Selenium, LLM evaluation frameworks, and prompt testing.
Experience developing agent‑powered test generation, workflow validation, or intelligent UAT assistants.
Experience with adversarial testing, Responsible AI testing, and production quality monitoring.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×