×
Register Here to Apply for Jobs or Post Jobs. X

AI Agent Evaluation Software Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), AI QA / Validation Engineer, Software Engineer, Software Testing
Job Description & How to Apply Below
Evaluate AI agents with software engineering and technical feedback skills. Outlier offers remote freelance work for contributors in listed countries.
This work supports leading AI organizations training large language models as proactive, multi-step agents. You will assess complex system behaviors and help improve real-world architectural workflows. The work suits developers with at least two years in backend engineering, AI automation, or complex systems integration, plus experience with live environments.

Key Responsibilities:

Debug agent traces across multi-turn interactions
Write detailed evaluation rubrics for agent performance
Stress test autonomous agents against complex workflows
Identify subtle failures in system behavior
Requirements:
At least two years of relevant professional experience
Maintain modular, production-grade software systems
Work effectively in live, non-mocked environments
Give high-density technical feedback on complex behaviors

Experience with backend engineering or AI automation
Use software engineering judgment, agent debugging, and evaluation skills to strengthen autonomous AI systems through Outlier projects.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary