×
Register Here to Apply for Jobs or Post Jobs. X

LLM Agent Evaluation Engineer

Job in Moncton, New Brunswick, Canada
Listing for: Outlier
Full Time position
Listed on 2026-11-04
Job specializations:
  • Software Development
    AI Engineer (Applied/Software), AI QA / Validation Engineer
Job Description & How to Apply Below
Evaluate LLM agents with Outlier using technical feedback, agent tracing, and stress testing. Work remotely as a freelance contributor on flexible projects.
Outlier helps innovative companies improve AI agents by providing human feedback. As an LLM Agent Evaluation Engineer, you will support systems designed for proactive, multi-step workflows. The work suits software developers and orchestration specialists with production experience, attention to detail, and complex integration skills.

Key Responsibilities:

Debug agent traces and investigate system behavior
Write rubrics for evaluating agent performance
Stress test autonomous agents against complex workflows
Requirements:
Two or more years of relevant professional experience
Experience building production-grade modular software
Practical multi-turn system interaction experience
Ability to deliver high-density technical feedback
Experience identifying privacy leaks and prompt injections
Use LLM evaluation, debugging, and systems expertise to improve autonomous agent performance with Outlier.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
 
 
 
Search for further Jobs Here:
(Try combinations for better Results! Or enter less keywords for broader Results)
Location
Increase/decrease your Search Radius (miles)
0
200
Filters
Education Level
Experience Level (years)
Posted in last:
Salary