Senior AI Quality Engineer

Position: Senior AI Quality Engineer Role
Join Rootly as a Senior AI Quality Engineer to refine and elevate AI functionalities. Focus on testing and optimizing AI systems to ensure peak reliability for users.

This role demands advanced expertise in QA and AI evaluation, with over 5 years of hands-on experience. As part of a dynamic team, you will design rigorous testing methodologies, assess AI outputs for alignment with workflows, and collaborate on feature acceptance criteria. Your insights will directly inform product quality and performance.

Key Responsibilities:

• Execute prompt-based tests for various AI scenarios

• Ensure AI outputs meet accuracy and relevance standards

• Develop structured evaluation frameworks for testing

• Document AI issues and work with teams on resolutions

• Establish and track key performance metrics

Requirements:

• 5+ years in QA, AI evaluation, or related fields

• Proven experience with agentic AI performance testing

• Strong prompt engineering skills and analytical thinking

• Clear communication skills for diverse audiences

• Experience with evaluation tools or frameworks

Help shape the future of AI in incident management at Rootly.
#J-18808-Ljbffr