AI Agent Evaluator, Task Testing; Remote
Listing for:
Vettedlance
Part Time, Remote/Work from Home
position
Listed on 2026-11-08
Job specializations:
-
Science
Job Description & How to Apply Below
Location: New York
Put AI agents through realistic multi-step jobs and judge whether they actually finish them. Your scenarios, ratings and notes show developers where an agent clicks wrong, skips a step or reports success it did not earn.
What you will do- Write realistic task scenarios for agents that browse, use apps or handle email and calendars
- Watch agent runs and record whether each step and the final result are correct
- Rate outcomes against a rubric and explain every failure in plain language
- Research claims and check data the agent used or produced
- Strong analytical and logical-reasoning skills
- Clear written English and careful attention to detail
- Comfort with everyday web apps, spreadsheets and productivity tools
- Able to follow detailed guidelines and work independently
- Remote, hourly contract (independent contractor)
- Part-time, flexible: 10-40 hrs/week
- Remote
Pay: $20 - $40 / hour (USD). Paid weekly in USD. Typical rate $20-$40 per hour, confirmed after a short paid assessment.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction