Software Engineer Task Author

About Open Train

Open Train AI is the hiring and contracting organization for this role and the #1 platform for finding and building careers in AI training and data labeling. Open Train helps people discover opportunities, build a professional profile, and grow in a rapidly expanding field where human expertise shapes how advanced AI systems work.

• Free Open Train account
• Remote opportunities across the AI training industry
• A platform for building a lasting AI training career

AI training is the human side of building artificial intelligence. Engineers and other specialists create examples, evaluate model behavior, and define reliable standards that help AI systems understand and perform complex work. In this role, your software engineering judgment will help train and assess AI agents on realistic technical challenges.

• Contribute directly to cutting-edge AI development
• Use your professional software engineering expertise in a flexible format
• Work remotely with a task-based schedule

Open Train is recruiting a Software Engineer Task Author to design and calibrate engineering challenges for AI training. You will create realistic prompts, scoring rubrics, and task environments that test how well AI agents handle real software engineering situations.

This is a flexible, task-based hourly contract for working software engineers in Canada. The commitment is 20 or more hours per week, with potential for ongoing work and contract extension.

• Employment type:
  
  Contractor and part time
• Location:
  
  Remote within Canada
• Language:
  English
• Pay: $70-$120 USD per hour, depending on experience

You will create technically accurate tasks, validate their solutions, and establish clear standards for evaluating AI-generated work. Each challenge should reflect realistic engineering conditions and have correctness criteria that can be checked consistently.

• Author engineering prompts involving integration failures, log-based debugging, bug traces, and configuration fixes
• Write scoring rubrics with exact correctness criteria and verification methods
• Set up task environments containing realistic codebases, logs, alerts, and system states
• Solve each task yourself to validate its soundness and accuracy
• Calibrate task difficulty to challenge the model appropriately
• Review and correct AI-drafted task prompts or rubrics when provided

This role requires hands-on, industry-level software engineering experience and the ability to reason carefully about real systems, failures, and fixes. You should be comfortable both solving technical problems and explaining how successful outcomes can be verified.

• Working software engineer with hands-on, industry-level experience
• Fluency in Git, version control workflows, and code review practices
• Strong debugging skills, including reading logs, using observability tools, and tracing failures through a codebase
• Experience with infrastructure, integrations, and configuration code, including APIs, services, configuration files, and CI/CD
• Ability to write and verify a code fix in a live or simulated environment
• Ability to define precise, checkable correctness criteria for engineering outcomes

The following experience can help you create more authentic and challenging engineering scenarios, although it is listed as helpful rather than required.

• Enterprise SaaS, e-commerce, telecommunications, or industrial engineering experience
• Experience with monitoring and observability platforms such as Datadog, Pager Duty, or Grafana
• Familiarity with cloud-native architectures on AWS, GCP, or Azure

This is a remote, hourly contract position for engineers located across Canada. Work is flexible and task-based, with a commitment of 20 or more hours per week. Compensation ranges from $70 to $120 USD per hour depending on experience.

• Remote work from Canada
• Flexible task-based contract
• 20+ hours per week
• $70-$120 USD per hour
• Potential for ongoing work and contract extension