AI Evaluation Engineer at Dialpad

Join Dialpad as an AI Evaluation Engineer to enhance the performance of Agentic AI systems. This role focuses on validation strategies, metric development, and collaboration with cross-functional teams in Vancouver.

As part of the AI Evaluation team, your main responsibility will be owning evaluation coverage for our AI systems. You will design and execute validation strategies across various workflows, improving regression evaluations and metrics. Your work will ensure our agentic voice and chat solutions are ready for release through thorough analysis and testing.

Key Responsibilities:

• Design validation strategies for NLP and speech workflows

• Execute regression evaluations and A/B comparisons

• Develop LLM-judge metrics and calibration processes

• Monitor data annotation jobs for accuracy

• Collaborate with applied science and engineering teams

Requirements:

• Bachelor's or Master's in Computer Science or related field

• 3+ years in QA, test engineering, or model evaluation

• Experience with AI-driven product evaluation

• Strong analytical skills and documentation

• Familiarity with collaborative technical environments

Make your mark at Dialpad by utilizing your QA expertise to innovate AI technology solutions.
#J-18808-Ljbffr