Member of Technical Staff, Evals

The Member of Technical Staff, Evals builds evaluation methods and systems to measure and improve frontier AI models and agents. The role combines research and software development across benchmarks, reward and verifier systems, human data quality, and open-source work.
* Permanent* PhD or equivalent research experience
* PhD or equivalent research experience### Skills and expertise

PythonLLM evaluation AI agent evaluation

Benchmark design

Reward models

Programmatic verifiers

Reinforcement-learning environments

Scalable software development### Key responsibilities
* Design evaluation frameworks and benchmarks for frontier language models, agents, and multimodal models.
* Develop reward models, verifiers, graders, and systems for high-quality human data.
* Prototype and evaluate systems, diagnose failure modes, and publish findings through research and open-source work.## How this job compares

Compared with 90 open mid-level research jobs on October 10, 2026.Pay:
This job pays $200K to $350K a year. The 32 open mid-level research jobs that publish US pay start at $199K on average, so this job starts in line with them.

Similar open jobs:
We Love AI Jobs lists 90 open mid-level research jobs at 53 companies.

Remote work:
This job is remote. 12 of the 90 mid-level research jobs (13%) are remote.

AI in this role:
High. AI is the core of the work: building, training or deploying AI systems. The role directly develops evaluations, benchmarks, feedback systems, and data methods for frontier AI models and agents. Building and improving AI evaluation systems is the core of the job.## Full description

About Handshake Handshake's mission is to organize expert human knowledge to advance the AI economy. Handshake AI works directly with frontier labs on their most consequential data, evaluation, and post-training challenges, building the systems that turn expert human knowledge into the data and evaluations that make frontier models better.

You will work alongside engineers, researchers, operators, and builders from organizations including Scale AI, Meta, Google, Amazon, xAI, Notion, and Palantir—and help build the systems that make expert human knowledge useful for advancing AI.

The Role We are hiring a Member of Technical Staff, Evals to help define how frontier AI systems are measured, understood, and improved. This is a broad, high-ownership role for researchers who build.

You will partner with AI researchers, domain experts, and customers to develop new benchmarks, reward and verifier systems, agent-evaluation methodologies, and data-quality techniques. You will work on questions at the center of frontier AI progress: what should be measured, how to design evaluations that reflect real capability, how to create high-signal feedback, and how to build the environments and data systems that make those answers actionable.

Early members of the team will have unusual influence over our technical direction, operating culture, and the open-source software, benchmarks, and research products we build. We care more about demonstrated research capability, technical judgment, and a builder's mindset than a specific title, degree, or career path.

Location:

San Francisco preferred; we are open to exceptional candidates in other locations.

What you'll do
• Design and build evaluation frameworks, benchmarks, and methodologies for frontier LLMs, AI agents, multimodal models, and reinforcement-learning environments.
• Develop reward models, programmatic verifiers, graders, and other feedback systems that make model behavior measurable and improvable.
• Research what makes evaluations representative, difficult, reliable, and resistant to short cutting or reward hacking.
• Build systems for high-quality human data, including expert task…