AI Annotation Specialist Fully Remote

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco our investors include Benchmark General Catalyst Peter Thiel Adam DAngelo Larry Summers and Jack Dorsey.

Position: AI Session Annotator (English)
Type: Contract
Compensation: $24 $ 30/hour
Location: Remote

Role Responsibilities

• Review recorded sessions where users interact with an AI assistant via voice and camera.
• Compare the assistants responses to visible video content. Identify hallucinated or fabricated visual details.
• Score sessions on visual interpretation accuracy relevance response timing and spatial language use.
• Evaluate if the assistant appropriately warned users about health safety or financial risks.
• Write detailed comments supporting scores with concrete session evidence.
• Capture conversation turns in the intake form.

Qualifications

Must-Have

• Native or native-level English spoken and written.
• Clear precise written English.
• An Android phone with a working camera and microphone.
• Attention to detail and consistent rubric application.

Preferred

• Experience in data annotation linguistic QA content moderation or AI model evaluation.
• Training in linguistics translation or interpreting.
• Ability to recognize unnatural or non-native English.

Start Date

• Immediate; high volume of work expected between 26 to 31 Aug.

Application Process (Takes 2030 mins to complete)

• Submit your resume or relevant background.
• Complete a short calibration task on sample sessions assessed on scoring consistency and comment usefulness.

Resources & Support

• For details about the interview process and platform information please check:
• For any help or support reach out to:

PS:
Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.