×
Register Here to Apply for Jobs or Post Jobs. X

AI Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), Backend Developer, AI QA / Validation Engineer
Job Description & How to Apply Below
Position: AI Agent Evaluation Engineer Outlier
Evaluate AI agents for Outlier remotely, using backend engineering and system integration expertise to improve autonomous model behavior.
This freelance opportunity supports projects training large language models as proactive, multi-step agents. Contributors write evaluation rubrics, debug agent traces, and stress test systems for complex architectural workflows. The work suits engineers with two or more years in backend engineering, AI automation, or complex systems integration, including experience with production-grade software.

Key Responsibilities:

Write rubrics evaluating agent performance and behavior
Debug traces from autonomous agent workflows
Stress test agents against complex scenarios
Assess multi-step coordination and system interactions
Requirements:
Two-plus years in backend engineering or AI automation

Experience with modular production software architectures
Practical experience using live, non-mocked environments
Ability to deliver clear technical feedback
Attention to subtle system failures
Help Outlier improve autonomous agents by combining AI evaluation, backend engineering, and complex systems integration experience.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary