More jobs:
Job Description & How to Apply Below
Role Overview
We are looking for an experienced QA Lead to drive quality strategy for an AI-powered platform. This role focuses on API automation, LLM evaluation frameworks, and regulated product lifecycle validation. You will design test infrastructure, define measurable quality gates, and ensure AI outputs meet reliability, safety, and performance standards.
The position requires deep backend testing expertise combined with emerging AI validation practices in a structured, compliance-driven environment.
Key Responsibilities
- Design, automate, and execute API test suites
- (functional, contract, performance, resilience)
- Define LLM evaluation strategies including:
- golden datasets, hallucination checks, guardrails, latency KPIs
- Build automated evaluation pipelines tied to release cycles
- Create PDLM deliverables:
- verification plans, protocols, traceability matrices, risk-based coverage
- Integrate test infrastructure with Azure Dev Ops pipelines
- Implement quality gates and dashboard reporting
- Collaborate with Product, Clinical, and R&D teams
- Validate AI outputs across workflows and user roles
- Ensure compliance with regulated product lifecycle processes
Required Skills
- 8+ years in backend/API automation testing
- REST/JSON testing and authentication workflows
- Performance testing experience
- Strong CI/CD integration background
- Python or Type Script automation
- Postman/Newman, REST-assured
- PyTest or Jest
- k6 or JMeter performance testing
- Docker-based test environments
- Hands-on validation of ML/LLM systems
- Experience with AI evaluation datasets and guardrails
- Strong documentation and traceability discipline (PDLM/QMS)
Good to Have
- Azure Dev Ops pipeline expertise
- Healthcare standards knowledge (FHIR / HL7)
- Data privacy & security awareness
- Frontend testing exposure
Behavioral Expectations
- Ownership mindset in quality leadership
- Strong cross-functional collaboration
- Structured thinking in regulated environments
- Ability to define scalable QA frameworks
- Adaptability to rapidly evolving AI technologies
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×