Mathematics Terminal Task Evaluation Lead

Lead reviews of terminal-based mathematics tasks for correctness, reproducibility, challenge, and reliable grading. This remote contractor assignment includes mentoring trainers and improving benchmarks across numerical, applied, and computational mathematics.

100% Remote

Contract, part-time

Engagement

Remote

Location

Oct 23, 2026

Posted

You will lead a pod of trainers and review complete terminal-based task packages used to evaluate AI systems. Each package may include a problem statement, datasets, software environment, reference solution, and automated tests.

Your reviews will cover mathematical correctness, reproducibility, meaningful challenge, and grading reliability across numerical and applied mathematics, scientific computing, optimization, statistics, mathematical modeling, computational algebra, and formal verification.

• Check whether problems are well posed and whether algorithms handle convergence, stability, conditioning, discretization error, numerical tolerances, and edge cases.
• Evaluate graders with residuals, feasibility checks, optimality certificates, in variants, error bounds, or proof checking.
• Confirm that tools, libraries, solver versions, and dependencies are pinned so tasks can be reproduced.
• Find hard-coded solutions, test leakage, and other ways someone could bypass the intended computation.
• Give precise feedback, track revisions through closure, mentor trainers, allocate work, monitor quality, and maintain documentation for recurring issues and strong example tasks.

This is a remote contractor assignment lasting four weeks. The role is listed as entry level, but the work requires advanced mathematical, programming, and technical leadership skills.

• Pay:
  Not provided in the listing.
• Location:
  
  Worldwide and remote.
• Time: 20 or more hours per week.
• Experience:
  
  A Ph.D., postdoctoral experience, or equivalent advanced experience in mathematics, applied mathematics, statistics, or a related field.
• Programming:
  Strong skills in Python, Julia, C or C++, MATLAB or Octave, or a computer algebra system.
• Technical background:
  Hands-on experience with numerical methods, scientific computing, symbolic computation, optimization, computational statistics, or mathematical modeling.
• Systems skills:
  Strong Linux and terminal proficiency, with the ability to assess reproducibility, algorithms, proofs, computational results, and test reliability.
• Other requirements:
  Strong written communication and experience mentoring or leading small technical or research teams.
• Useful background:
  Num Py, Sci Py, Sym Py, mpmath, Sage Math, PARI/GP, GAP, FEniCS, Firedrake, Differential Equations.jl, PETSc, CVXPY, JuMP, Pyomo, proof assistants, Docker, Git, continuous integration, automated testing, benchmark design, computational evaluations, or AI coding and reasoning evaluations.
• Relevant evidence may include publications or open-source contributions in computational mathematics, numerical software, or formalized mathematics.

AI training work is the human side of building artificial intelligence. People create and review examples, tasks, and model evaluations so AI systems can produce more accurate and useful results; advanced mathematical expertise is valuable for checking difficult technical work.

Open Train is a platform where people find AI training and data-labeling projects, build a profile, and apply in minutes. Creating an account is free.

• Experience:
  
  Entry level