Production AI Evaluation Specialist

About Open Train

Open Train is the #1 platform for finding and building careers in AI training and data labeling. We connect experienced professionals with specialized opportunities where their practical judgment helps shape how modern AI systems are evaluated and improved.

Creating an Open Train account is free. You can build a profile that showcases your experience, discover relevant projects, and apply in minutes while developing a lasting portfolio of AI training work.

AI training is the human side of building artificial intelligence. Professionals create examples, review model outputs, and design structured evaluations that help AI systems perform more reliably in real-world situations.

In this role, your production expertise will be translated into realistic scenarios and assessment criteria. Your work will help evaluate whether AI-generated decisions properly account for budgets, schedules, contracts, vendors, crews, and other production constraints.

Open Train is seeking a film, television, digital, or live event production expert to create realistic evaluation tasks for AI training. You will convert production-management judgment into structured scenarios involving budget reconciliation, scheduling adjustments, contracts, call sheets, delivery planning, vendor coordination, and crew assignments.

This is a part-time contractor engagement requiring 20 or more hours per week. The work is remote and asynchronous, with compensation ranging from $45 to $85 USD per hour.

• Category:
  Production AI evaluation and text generation
• Experience:
  
  Five or more years of credited production experience
• Schedule:
  
  20 or more hours per week
• Engagement:
  Part-time contractor
• Language:
  English
• Compensation: $45 to $85 USD per hour

You will design tasks that simulate the practical decisions production professionals make under competing constraints. Each task should reflect realistic industry workflows and provide enough context to assess nuanced reasoning and deliverable quality.

You will work with project collaborators to refine task designs while maintaining credible production standards. Documentation must be clear and precise enough for AI benchmarking and technical review.

• Design production-management evaluation scenarios, including budget reconciliation and scheduling adjustments.
• Source, construct, and curate authentic budgets, contracts, call sheets, delivery schedules, and crew assignments.
• Define correct decisions and comprehensive solutions that account for multiple production constraints.
• Develop rubrics with 35 or more criteria for evaluating deliverables, judgment, and constraint management.
• Document production processes and workflows in clear technical English.
• Refine evaluation tasks with collaborators while preserving realistic industry standards.

You should have substantial hands‑on production experience and be comfortable making decisions that balance time, cost, resources, and operational realities. The role requires both professional production judgment and the ability to express that judgment through structured assessments.

Strong written and verbal English communication is essential because you will create technical materials and document workflows for evaluation and benchmarking.

• At least five years of credited experience in film, television, digital, or live event production.
• Direct experience managing and reconciling production budgets, call sheets, and contracts.
• Ability to coordinate vendors and crew while balancing time, cost, and resource constraints.
• Experience creating realistic production artifacts, defining correct decisions, or evaluating multi‑criteria rubrics.
• Strong written and verbal communication skills for production documentation…