×
Register Here to Apply for Jobs or Post Jobs. X

AI Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI QA / Validation Engineer
Job Description & How to Apply Below
Evaluate AI agents remotely for Outlier using technical feedback, trace debugging, and stress testing across real-world interactions.
Outlier collaborates with AI organizations to train Large Language Models as proactive, multi-step agents. This freelance work centers on assessing how systems design, coordinate, and optimize complex workflows. Candidates need at least two years of relevant experience and strong attention to subtle system behaviors.

Key Responsibilities:

Stress test autonomous agents across demanding scenarios
Debug traces from multi-step system interactions
Write clear, precise evaluation rubrics
Identify subtle failures in agent behavior
Requirements:
Two or more years in relevant technical fields
Production-grade software development with modular separation
Experience handling live environments without mocks
Clear technical feedback on complex system behavior
Applicants must be authorized in their residence country
Use evaluation, debugging, and technical analysis skills to improve autonomous AI agents through Outlier’s remote freelance projects.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary