AI Evaluation Engineer; Fully Remote

• Design and build evaluation infrastructure for production-grade AI agents.
• Develop automated testing systems for complex reasoning and multi-turn conversations.
• Own the evaluation architecture from ground up in a fully remote environment.

Key Responsibilities:

• Build CI/CD testing pipelines to validate AI agent behavior.
• Create statistical evaluation frameworks for non-deterministic AI behavior.

Requirements:

• Strong proficiency in Python or Type Script.
• Experience building LLM evaluation pipelines and integrating tests into CI/CD.
• Excellent English and ability to overlap with US Central Time business hours.
• Fully remote work with flexible schedule.
• Paid training and professional development opportunities.
• Performance-based annual pay increases and negotiable hourly rate.

A company building production-grade AI agents and systems. They operate in a fully remote, globally distributed environment with a culture focused on innovation, autonomy, and continuous improvement.

We have an opening for a AI Evaluation Engineer (Fully Remote) in United States within IT & Technology, Engineering.