AI Agent Evaluator - Remote
Listing for:
YO AI Labs
Remote/Work from Home
position
Listed on 2026-11-21
Job specializations:
-
Business
Job Description & How to Apply Below
Title:
AI Agent Evaluator
Role Type: Contractor
Location: United States
We are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. You'll test two AI agents side by side, assess their performance, and provide feedback to improve their reliability and user experience.
Key Responsibilities- Test AI agents on everyday tasks, including shopping, travel, bookings, bill payments, email follow-ups, and forms.
- Approve or stop agents at key steps.
- Screen-record each testing session.
- Score task completion, quality, control issues, user satisfaction, and time.
- Provide clear written feedback on agent performance and usability.
- Based in the United States with a bachelor's degree.
- Regularly use AI tools in daily life.
- 2–3 years of experience, preferably in engineering, business management, or office operations.
- Strong attention to detail and ability to follow instructions.
- Excellent written communication and documentation skills.
- An active personal AI account (Free, Go, or Plus; not Pro) and a smartphone capable of screen recording.
Preferred: Relevant domain expertise. No coding or AI development experience is required.
Project Details- Task duration: Approximately 1.5 hours per run, including evidence and scoring.
- Kickoff: Monday, October 12.
- Onboarding: Before the project begins, subject to scheduling.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction