LLM Annotator - Master's Degree · Turing Turing · Varies · remote · ago Varies ago

Turing is seeking detail-oriented LLM Annotators to help evaluate and improve cutting-edge Large Language Models. This is a short-term remote contract role ideal for analytical professionals with advanced degrees who enjoy working at the frontier of AI research.

Based in San Francisco, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. Turing helps accelerate frontier research with high-quality data, advanced training pipelines, and top AI researchers specializing in coding, reasoning, STEM, multilinguality, multimodality, and agents.

• Design challenging prompts that evaluate an LLM's ability to retrieve, analyze, and reason over structured data.
• Assess AI-generated responses for factual accuracy, logical reasoning, and completeness.
• Identify model failures, hallucinations, inconsistencies, and reasoning gaps.
• Validate model outputs using provided datasets and supporting evidence.
• Document findings with clear, evidence-based explanations.
• Adhere consistently to annotation guidelines and maintain high-quality standards.

• Master's degree or higher in any discipline.
• Minimum 3 years of professional, research, or teaching experience.
• Strong analytical and critical thinking skills.
• Excellent written English communication skills.
• Exceptional attention to detail and ability to validate information against source data.

• Experience working with Large Language Models (LLMs) or Generative AI.
• Familiarity with prompt engineering, AI evaluation, data annotation, or model testing.
• Experience with structured datasets (CSV, Excel, databases, etc.).
• Ability to identify edge cases and design prompts that expose model limitations.

• Commitment: 20, 30, or 40 hours per week with at least 4 hours PST overlap required.
• Employment Type: Contractor assignment (no medical/paid leave benefits).
• Duration: 4-week contract.

• Work on cutting-edge AI research projects with real-world impact.
• Competitive compensation with flexible, remote working hours.
• Collaborate with leading AI researchers and engineers globally.