Astronomy and Space Science AI Evaluator

About Open Train

Open Train AI is the hiring and contracting organization for this opportunity. Open Train is the #1 platform for finding and building careers in AI training and data labeling, helping contributors discover projects, build an AI training profile, and apply in minutes.

In this role, your astronomy and space science expertise will help improve how advanced AI systems reason about scientific questions and communicate technical conclusions.

• Remote contract engagement
• Part-time opportunity with a commitment of 20+ hours per week
• Open to qualified contributors in participating countries
• English-language work

AI training is the human side of building modern artificial intelligence. Specialists review model responses, test reasoning, identify errors, and provide structured feedback so AI systems can produce more accurate and reliable outputs.

Scientific evaluators are especially important when models handle calculations, technical explanations, uncertainty, and limits of inference. Your judgment will help shape AI that communicates astronomy and space science responsibly.

• Review AI-generated responses and step-by-step solutions
• Evaluate prompts, model answers, and technical reasoning
• Identify overconfident claims, missing constraints, and misleading conclusions
• Help models communicate assumptions, uncertainty, and limitations clearly

Open Train is seeking an experienced astronomy or space science specialist to evaluate AI-generated content across a broad range of scientific topics. You will assess scientific accuracy, logical soundness, completeness, clarity, units, assumptions, approximations, uncertainty, and limits of inference.

The work is designed for contributors who can apply strong quantitative reasoning and provide high-quality scientific judgment on advanced model outputs. You will challenge models with realistic scenarios, calculations, and research-style reasoning tasks.

• Focus on scientific evaluation of AI outputs
• Apply domain-specific reasoning in astronomy and space science
• Provide clear, structured feedback on technical accuracy and uncertainty
• Work with text-based AI training and evaluation tasks

You will review and refine scientific prompts, model responses, and solutions while checking whether the reasoning is valid and appropriately qualified. Your feedback will address both conceptual and quantitative problems.

• Evaluate AI-generated responses for scientific accuracy and logical soundness
• Review and refine prompts, model answers, and step-by-step solutions
• Check units, assumptions, approximations, uncertainty, and limitations
• Identify conceptual errors, bad assumptions, missing constraints, and formula mistakes
• Assess work involving orbital mechanics, stellar and galactic astrophysics, cosmology, planetary science, space physics, and data interpretation
• Use orders of magnitude, unit consistency, limiting cases, and error propagation to validate results
• Challenge overconfident scientific claims and explain responsible limitations
• Give structured scientific feedback on model answers

This role requires a substantial professional background in astronomy or a closely related space science field, along with the ability to evaluate quantitative work carefully. A bachelor's degree or higher is required, with advanced study strongly preferred.

• 4+ years of professional experience in astronomy or a closely related space science domain
• Bachelor's degree in physics, astronomy, space science, engineering, or a closely related field
• Master's degree or PhD strongly preferred
• Strong command of orbital mechanics, astrophysics, cosmology, planetary science, or space physics
• Deep knowledge of relevant fundamentals, including radiative processes, stellar and galactic physics, coordinate systems, time standards, statistics, and scientific modeling assumptions
• Strong quantitative reasoning and scientific sanity-checking skills
• Ability to check units, uncertainty, assumptions, and feasibility in model outputs
• Ability to communicate limits responsibly and challenge overconfident scientific claims

Experience with AI data training or technical content evaluation is helpful, but the central requirement is the ability to make careful, domain-specific scientific judgments.

• Experience reviewing quantitative astronomy or space science work
• Experience evaluating AI-generated technical explanations
• Experience with AI data training, annotation, or red-teaming
• Familiarity with Python, Num Py, Sci Py, data pipelines, catalog queries, photometry, spectroscopy, or mission datasets
• Background in orbit determination, photometry, spectroscopy, or related quantitative analysis
• Comfort working with scientific modeling assumptions and uncertainty

This is a part-time contractor opportunity with remote collaboration and a 20+ hour weekly commitment. Compensation varies by location, with…