AI Rater Guidelines Writer

About Open Train

Open Train is the #1 platform for finding and building careers in AI training and data labeling. Open Train AI is hiring and contracting for this opportunity, helping contributors find meaningful projects, build a professional AI training profile, and apply in minutes.

• Free account creation
• Access to opportunities across the growing AI training industry
• A way to build experience and a lasting AI training career

AI training is the human work behind modern artificial intelligence. People write, review, rate, and refine examples that help language models produce more accurate, useful, and reliable responses.

As an AI Rater Guidelines Writer, you will contribute at the instruction-design layer of this process. Your guidelines and rubrics will help human raters evaluate model outputs consistently, including when examples are ambiguous or difficult to classify.

• Work on cutting-edge generative AI and RLHF projects
• Help shape how advanced language models are evaluated
• Apply writing, reasoning, and domain-translation skills to practical AI development

Open Train AI is seeking an AI Rater Guidelines Writer to turn high-level, sometimes ambiguous program specifications into precise instructions for human raters. You will bridge program requirements and field application by creating clear, non-contradictory guidelines that support consistent decisions across diverse subject-matter domains.

This is a part-time contractor opportunity for applicants in the United States who work in English. The listed rate is $45-$65 per hour.

• Role: AI Rater Guidelines Writer
• Work type:
  Contractor and part time
• Location:
  
  United States
• Language:
  English
• Pay: $45-$65 per hour
• Subject matter:
  Cross-domain rater guidelines

You will collaborate with research, program, and subject-matter teams to make evaluation instructions accurate, usable, and consistent. The work includes both initial guideline development and detailed quality review before instructions are used in the field.

• Translate high-level and ambiguous program specifications into precise, unambiguous rater guidelines.
• Design rating rubrics that raters can apply consistently, including for edge cases.
• Review draft guidelines for internal contradictions and coverage gaps.
• Revise guidelines until they are ready for field application.
• Adapt instructions to terminology and standards in finance, retail, insurance, legal, and sports domains.
• Collaborate with subject-matter experts and program leads to maintain consistency and accuracy across the full guideline set.

This role requires professional experience developing clear written guidance and direct experience supporting human rating in a generative AI or RLHF environment. You should be comfortable identifying ambiguity, resolving contradictions, and explaining complex requirements in straightforward language.

• 3+ years of professional experience in linguistics, instructional design, technical writing, or a closely related field.
• Direct experience writing or refining guidelines and rubrics for human raters in a GenAI or RLHF environment.
• Ability to work across multiple subject-matter domains and translate domain-specific nuance into clear instructions.
• A strong track record of resolving ambiguity and contradiction in written specifications.
• Strong written communication skills, including the ability to explain complex guidance clearly.
• Experience with rating guidelines in fields such as finance, insurance, legal, retail, or sports.
• Availability for weekday work.
• The role details specify at least 35 hours per week; the structured listing indicates a 20+ hour weekly requirement.

This opportunity is suited to a precise, analytical writer who enjoys turning complicated requirements into practical instructions. It may be a strong fit for experienced technical writers, instructional designers, linguists, or AI evaluation professionals who can move confidently between domains without losing important nuance.

The listing identifies the experience level as entry level, while the detailed requirements call for 3+ years of relevant professional experience and prior GenAI or RLHF guideline work. Applicants should review those requirements carefully before applying.

• You write with precision and care about consistency.
• You can anticipate edge cases and explain how raters should handle them.
• You are comfortable reviewing specifications for gaps and contradictions.
• You can adapt language for different industries and subject-matter standards.
• You communicate effectively with research, program, and subject-matter teams.