×
Register Here to Apply for Jobs or Post Jobs. X

Autonomous Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), AI QA / Validation Engineer
Job Description & How to Apply Below
Evaluate autonomous AI systems through agent tracing, stress testing, and rubric writing. This remote freelance work supports Outlier projects for advanced generative systems.
Outlier collaborates with leading AI organizations to improve large language models as proactive, multi-step agents. Applicants need two or more years in backend engineering, AI automation, or complex systems integration. The work centers on production-grade software, live tools, persistent state, and multi-turn interactions.

Key Responsibilities:

Create evaluation rubrics for complex agent tasks
Investigate failures within agent execution traces
Stress test autonomous agents against demanding workflows
Assess system behaviors with high-density technical feedback
Requirements:
Two or more years of applicable engineering experience
Maintain modular, production-grade software architectures
Build for live environments rather than mocked systems
Manage multi-turn system interactions carefully
Identify privacy leaks, authority escalation, or prompt injections
Bring software engineering and AI automation experience to rigorous agent evaluation work involving Open Claw and real-world workflows.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary