AI Evaluation Engineer

Transform AI evaluation at Dialpad, a leading customer experience platform, as an AI Evaluation Engineer in Vancouver. You will focus on validation strategies, LLM metrics development, and impactful data analysis.

In this vital role, you will co-own evaluation initiatives for Dialpad's Agentic AI systems, contributing to product readiness and quality assurance. Responsibilities include designing validation strategies across various workflows, executing regression evaluations, and ensuring calibration datasets are well-managed. Your analytical skills will drive improvements in AI products while collaborating with cross-functional teams.

Key Responsibilities:

• Design validation strategies for NLP and speech workflows

• Execute regression evaluations and A/B comparisons

• Co-own LLM metric development and calibration

• Create and monitor data annotation jobs

• Investigate and triage bugs and issues

Requirements:

• Bachelor’s or Master’s in Computer Science or related field

• 3+ years in QA, test engineering, or model evaluation

• Experience with complex AI systems

• Strong analytical skills with cross-functional teams

• Proficiency in designing structured test strategies

Contribute to innovative AI solutions at Dialpad, enhancing customer experience through thorough evaluation processes.
#J-18808-Ljbffr