More jobs:
Job Description & How to Apply Below
Drive AI excellence at Rootly as an AI Quality Engineer. This pivotal role focuses on optimizing agentic AI features through rigorous testing, evaluation, and collaboration with engineering teams.
In this senior role, you'll leverage your 5+ years of experience in QA and AI/ML evaluation to design and execute comprehensive test scenarios for Rootly's innovative platform. Your responsibilities include building evaluation frameworks, identifying AI performance issues, and collaborating closely with Product and Engineering on new feature releases.
Key Responsibilities:
• Design prompt-based test scenarios for AI features
• Evaluate AI outputs for accuracy and consistency
• Construct and maintain test libraries and scoring rubrics
• Identify and document failure modes and unexpected behaviors
• Define quality metrics and report findings to stakeholders
Requirements:
• 5+ years in QA or AI/ML evaluation roles
• Hands-on experience with LLM-powered AI products
• Strong analytical and communication skills
• Familiarity with evaluation frameworks and adversarial testing
• Comfort with Python for scripting and evaluation tools
Elevate the quality of AI interactions as you influence Rootly's incident management innovations.
#J-18808-Ljbffr
Note that applications are not being accepted from your jurisdiction for this job currently via this jobsite. Candidate preferences are the decision of the Employer or Recruiting Agent, and are controlled by them alone.
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
To Search, View & Apply for jobs on this site that accept applications from your location or country, tap here to make a Search:
Search for further Jobs Here:
×