AI - QA & Prompt Analyst
Job in
Boston, Suffolk County, Massachusetts, 02298, USA
Listed on 2026-09-21
Listing for:
Trinityaiboston
Full Time
position Listed on 2026-09-21
Job specializations:
-
IT/Tech
AI Engineer (Applied/Software), IT QA Tester / Automation
Job Description & How to Apply Below
We are looking for an AI QA & Prompt Analyst to ensure the quality, reliability, and performance of our AI models and systems. You will design test strategies for LLM-based applications and work closely with our AI engineering teams.
What You’ll Do- Design and execute test plans for LLM-based applications and AI systems
- Develop and optimize prompts for various AI use cases
- Identify and document AI model limitations, biases, and failure modes
- Create automated testing frameworks for AI quality assurance
- Collaborate with AI engineers to improve model performance
- 3+ years of QA experience with at least 1 year in AI/ML testing
- Strong understanding of LLMs and prompt engineering principles
- Experience with Python for test automation
- Familiarity with AI evaluation metrics and methodologies
- Excellent analytical and problem-solving skills
- Experience with MLflow, Lang Chain, or similar frameworks
- Knowledge of RAG architectures and vector databases
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
Search for further Jobs Here:
×