AI Agent Evaluator
Job in
Boise, Ada County, Idaho, 83701, USA
Listing for:
Anyone AI
Part Time
position
Listed on 2026-11-05
Job specializations:
-
Science
Job Description & How to Apply Below
- Location:
Remote - Type:
Independent contractor / Part-time - Project duration: up to 3 months
What You’ll Do
- Write prompts about questions, interests, or needs relevant to you.
- Have natural, multi-turn conversations with two AI assistants.
- Include questions that draw on information or activity from your connected services, where relevant.
- Compare how effectively each assistant uses available information and connects relevant context.
- Assess whether responses feel helpful, natural, trustworthy, and safe.
- Evaluate how each assistant responds to your feedback and corrections.
- Provide clear explanations supporting your evaluations.
- Fluency in one of the supported languages/locales listed below.
- Strong reading comprehension and the ability to explain your feedback clearly.
- Attention to detail and sound judgment when comparing conversation experiences.
- Willingness to interact with AI assistants using relevant personal context and connected services.
- Recent activity across a qualifying combination of digital services.
- Completion of the project’s eligibility verification and corresponding expert requirements.
- Spanish — Latin America
- Arabic
- Portuguese — Brazil
- Japanese — Japan
- Turkish — Turkey
- Indonesian — Indonesia
- Hindi — India
- Korean — Korea
- French — France
- German — Germany
Completing verification does not guarantee placement. Experts who do not qualify for this project may still be considered for other AI evaluation opportunities. Nice to have
- Experience using AI assistants for everyday questions, planning, or research.
- Previous experience in AI evaluation, user research, or quality assurance.
- Ability to identify subtle differences in relevance, personalization, and conversation quality.
To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction