Senior Automation Tester in Python; Gen AI powered
Job in
Phoenix, Maricopa County, Arizona, 85001, USA
Listed on 2026-10-30
Listing for:
EPAM Systems
Full Time
position Listed on 2026-10-30
Job specializations:
-
Quality Assurance - QA/QC
AI QA / Validation Engineer, IT QA Tester / Automation
Job Description & How to Apply Below
We are looking for a Senior Automation Tester in Python to join Production Pods within the program. This is not a manual/functional QA role. You will build and maintain automated test frameworks for AI/GenAI-powered clinical applications, validate non-deterministic AI outputs, and ensure GxP compliance through automated IQ/OQ/PQ qualification. You will work across multiple use cases in Safety, Regulatory, Clinical Operations, and SAGE platform tooling.
Responsibilities- Build automated test frameworks from scratch for AI-powered clinical applications using Playwright, Selenium, and pytest
- Design and execute test strategies for non-deterministic AI/GenAI outputs, including accuracy, RAG relevance, hallucination checks, and prompt regression
- Integrate automated tests into CI/CD pipelines with quality gates such as block-on-failure and coverage thresholds
- Execute GxP-compliant IQ/OQ/PQ scripts and generate qualification evidence packages
- Manage AI-specific test data, including edge cases, adversarial inputs, and golden datasets
- Monitor model drift, regression, and performance degradation over time
- Produce test strategy documents, test plans, and compliance-ready test evidence
- Collaborate with Data Scientists and Backend Developers to define testable acceptance criteria for AI features
- 5+ years of hands-on QA/test automation experience
- Expertise in building and maintaining automated test suites from scratch using Playwright, Selenium, Cypress, and pytest
- Skills in API and backend test automation, including REST APIs, microservices, and serverless architectures with Postman/Newman and pytest + requests
- Proficiency in Python as the primary automation language, with the ability to write harnesses, fixtures, and utilities independently
- Knowledge of CI/CD test integration, embedding automated tests into pipelines such as Jenkins and Git Hub Actions with quality gates
- Background in testing AI/GenAI applications, validating non-deterministic AI/LLM outputs for accuracy, RAG relevance, hallucination, and prompt regression
- Familiarity with AI/ML model testing, including accuracy, bias, and performance validation using RAGAS, LLM-as-judge, and golden datasets
- English proficiency at B2 level or higher
- Nice to have Background in pharmaceutical / life sciences domain, with GxP awareness and IQ/OQ/PQ qualification
- Experience with Jira/Xray, Test Rail, or Zephyr test management
- Skills in performance/load testing using JMeter and Locust
- Understanding of security testing basics such as OWASP
- Familiarity with AWS services including Lambda, API Gateway, S3, and Cloud Watch
Position Requirements
10+ Years
work experience
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×