×
Register Here to Apply for Jobs or Post Jobs. X

Autonomous AI Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), AI QA / Validation Engineer
Job Description & How to Apply Below
Evaluate autonomous AI agents with Outlier using technical feedback, agent debugging, and complex systems integration. This freelance work is remote.
Outlier helps innovative companies improve AI agents by collecting human feedback. As an engineer, you will help train large language models to operate as proactive, multi-step systems. Projects involve architectural workflows, live tools, and system behaviors requiring careful analysis.

Key Responsibilities:

Debug agent traces and investigate system behavior
Write clear evaluation rubrics for agent outputs
Stress test agents against complex interactions
Identify privacy leaks and authority escalation failures
Review indirect prompt injection risks
Requirements:
At least two years of relevant professional experience
Backend engineering, AI automation, or integration background
Maintain production-grade modular software systems
Handle persistent state and multi-turn interactions
Provide precise technical feedback on complex behaviors
Use agent evaluation, debugging, and security analysis skills to help Outlier improve autonomous AI systems like Open Claw.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary