×
Register Here to Apply for Jobs or Post Jobs. X

Autonomous Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), AI QA / Validation Engineer
Job Description & How to Apply Below
Evaluate autonomous agents through rubrics, trace debugging, and stress testing with Outlier as a remote freelance contributor.
Outlier collaborates with leading AI organizations to improve large language models and proactive, multi-step agents. This opportunity is designed for professionals with two or more years in backend engineering, AI automation, or complex systems integration. Your feedback will address complex system behaviors across real-world workflows.

Key Responsibilities:

Create evaluation rubrics for agent behavior
Debug traces from autonomous agent interactions
Stress test agents in live environments
Assess multi-turn system interactions carefully
Identify subtle failures in agent behavior
Requirements:
Two or more years in backend or AI work
Production-grade software development experience

Experience with modular service separation
Ability to provide high-density technical feedback
Recognize privacy leaks and prompt injections
Bring precise evaluation, debugging, and systems expertise to help Outlier improve advanced generative agent systems.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary