AI Agent Evaluator, Task Testing; Remote

Position: AI Agent Evaluator, Real-World Task Testing (Remote)

Put AI agents through realistic multi-step jobs and judge whether they actually finish them. Your scenarios, ratings and notes show developers where an agent clicks wrong, skips a step or reports success it did not earn.

What you will do

• Write realistic task scenarios for agents that browse, use apps or handle email and calendars
• Watch agent runs and record whether each step and the final result are correct
• Rate outcomes against a rubric and explain every failure in plain language
• Research claims and check data the agent used or produced

What you need

• Strong analytical and logical-reasoning skills
• Clear written English and careful attention to detail
• Comfort with everyday web apps, spreadsheets and productivity tools
• Able to follow detailed guidelines and work independently

Contract

• Remote, hourly contract (independent contractor)
• Part-time, flexible: 10-40 hrs/week
• Remote

Pay: $20 - $40 / hour (USD). Paid weekly in USD. Typical rate $20-$40 per hour, confirmed after a short paid assessment.