Applied Research Engineer

Develop research-informed methods that make it easier to evaluate applications built with large language models. The work spans optimization, synthetic test data, and adapting models to serve as evaluators, with a focus on making automated assessments align more closely with human judgments.

• Design optimization algorithms to improve agreement between model-based evaluators and human assessments.
• Create methods for generating synthetic test cases for language model applications, including retrieval-augmented and tool-using workflows.
• Fine-tune and align language models for use as custom evaluators.
• Apply findings from research to practical evaluation problems and develop new solutions.

• Extensive experience working with large language models.
• Experience with synthetic data generation and optimization.
• Interest in reading and critically assessing research while remaining open to new approaches.
• Evidence of interest in open source or competitive data science, such as contributions to open-source projects or Kaggle participation.

• Remote role.
• Advertised compensation range: $60,000–$200,000, plus 0.5%–2.5% equity.