AI Side by Side Quality Evaluator Remote

($16.66 /hr - Remote)

We’re looking for English-fluent experts to engage on a project about the overall quality of AI-generated responses.

As an independent contractor, you’ll interact directly with large language models (LLMs) and compare their responses side by side to decide which one better meets the user’s needs. Your evaluations and written rationales will help build AI systems that are more helpful, accurate, and reliable.

Experts may be asked to:

• Interact directly with different AI models and tools

• Compare two or more model responses side by side (SxS)

• Judge responses on several quality dimensions, such as accuracy, helpfulness, instruction-following, clarity, and overall quality

• Follow multi-step evaluation workflows and detailed rating guidelines

• Write clear, well-reasoned explanations for each evaluation decision

• Record consistent judgments across a wide range of prompts and topics

This project may be a strong match for experts who:

• Are fluent in English, with strong reading comprehension and writing skills

• Have extensive hands-on experience using LLMs and evaluating their outputs

• Have strong critical-thinking skills and can compare responses on several quality dimensions

• Pay close attention to detail and follow instructions carefully

• Can explain their evaluation decisions clearly and convincingly

• Are comfortable following multi-step workflows and switching between different models and tool

• Contractor's services are provided on an independent contractor basis only and do not create any right or expectation of continued engagement or employment.
• Projects may be extended, shortened, or concluded early, including for reasons tied to project needs and the quality of deliverables; nothing in this engagement guarantees any minimum amount of work.
• Any engagement is fully remote, can be completed on your own schedule, and contractors are free to accept or reject any project opportunity.
• Payment for services provided will be issued weekly via Stripe or Wise.
• We are unable to support H1-B or STEM OPT contractors at this time.

Meridial connects you with remote, paid AI projects that match your skills and expertise. By contributing your knowledge, you help advance the world's leading AI systems and accelerate the future of artificial intelligence. Hear from Charlie, to learn more.