Research Scientist, Human AI Interaction

About Handshake

Handshake is the career network for the AI economy. 20 million knowledge workers, 1,600 educational institutions, 1 million employers (including 100% of the Fortune 50), and every foundational AI lab trust Handshake to power career discovery, hiring, and upskilling, from freelance AI training gigs to first internships to full-time careers and beyond. This unique value is leading to unparalleled growth; in 2025, we tripled our ARR at scale.

Why join Handshake now:

• Shape how every career evolves in the AI economy, at global scale, with impact your friends, family and peers can see and feel

• Work hand-in-hand with world-class AI labs, Fortune 500 partners and the world’s top educational institutions

• Join a team with leadership from Scale AI, Meta, xAI, Notion, Coinbase, and Palantir, among others

• Build a massive, fast-growing business with billions in revenue

As a Research Scientist, Human–AI Interaction
, you will play a pivotal role in defining how AI systems support real human work by leading research at the intersection of Human–Computer Interaction (HCI),
Large Language Models (LLMs), and task-level benchmarking
.

You will operate at the frontier of human-centered AI evaluation
, with a focus on understanding what people actually do to accomplish meaningful work—and how AI systems change, accelerate, or reshape that activity. Your research will define jobs-to-be-done benchmarks
, comparative evaluation frameworks, and empirical methods for measuring human effort, time, quality, and outcomes when working with AI copilots. Additionally, the Handshake AI platform is an interface used by thousands of the top subject matter experts in the world to evaluate AI systems, and offers numerous interesting HCI / HITL-AI research questions that will drive large business impact.

You’ll set research direction, establish standards for measuring human activity in AI-mediated workflows, publish papers and open-source code, and lead the development of rigorous, scalable benchmarks that connect human work
, AI assistance
, and real economic value
.

• Lead high-impact research on jobs-to-be-done benchmarks for AI systems
  , including:

  • Defining task taxonomies grounded in real professional and economic activities

  • Identifying what constitutes meaningful task completion, quality, and success

  • Translating qualitative work understanding into measurable, repeatable benchmarks

• Develop methods to measure human activity in AI-mediated workflows

• Design benchmarks to assess AI-as-a-collaborator/copilot, rather than autonomous agents / basic Q&A

• Design and run empirical studies of how people use AI to solve tasks
  , including:

  • Controlled experiments and field studies measuring task performance

  • Instrumentation for capturing fine-grained interaction traces and outcomes

• Drive strategy for professional-domain AI benchmarks
  , focusing on:

  • Understanding domain-specific workflows (e.g., analysis, writing, planning, coordination)

  • Grounding benchmark design in how work is actually performed, not idealized tasks

• Build and prototype AI systems and evaluation infrastructure to support research and Data production, including:

  • LLM-powered copilots and experimental tools used for task-level measurement

  • Benchmark harnesses that evaluate both model behavior and human outcomes

  • Data pipelines for analyzing human–AI interaction at scale

  • The human-in-the-loop experience for Handshake fellows to produce effective evaluations and training data for frontier models, through structured UI/UX interactions with these models.

• Collaborate closely with User Experience Research (UXR) to:

  • Leverage deep qualitative insights into real user behavior and workflows

  • Translate ethnographic and observational findings into formal research constructs

(This role is distinct from UXR and focuses on measurement, modeling, and evaluation.)

• Publish and present research that advances the field of human-centered AI benchmarking, with an expectation of regular contributions to top-tier venues such as CHI (Conference on Human Factors in Computing Systems), and related HCI and AI conferences.

• PhD or equivalent experience in Human–Computer Interaction, Computer…