Model Behavior Engineer, Reasoning and Decision Making

Elicit helps research and business teams investigate hard questions and make high-stakes decisions grounded in evidence.

Our ambition is to make high-quality reasoning under uncertainty available at a much greater scale in contexts where stakes are high and feedback is sparse. We're building AI systems whose work people can inspect, question, and improve, so that increasingly capable AI also makes us more wise.

Some of the most valuable questions people face are also the hardest to answer well. Which research program should a team advance? What evidence would change its priorities? Which uncertainty is worth resolving before committing to a multi-year, $100 million dollar trial?

Today's models can contribute to these decisions, but they often miss a crucial consideration, make an assumption they should have checked, or reverse a recommendation too readily when presented with new information. Further, because they can perform increasing amounts of work autonomously, their process and reasoning is often poorly legible, so decision makers are wary to trust their outputs. Helping with these problems requires a deep understanding of model behaviors, what matters for different types of decisions, the environments and cultures in which decisions are made, and how to evaluate complex, multiplayer systems with many human and agent interactions.

We're looking for someone to own that work: understand where our systems fall short, develop ways to evaluate them, and build improvements that help users make better decisions. You'll start with concrete problems in scientific R&D and commercial strategy, and develop approaches that apply across different kinds of decisions.

You'll join our engineering team and report to an Engineering Manager. You'll work on a cross-functional engineering team, collaborating closely with other engineers, product, design, and domain experts, taking problems from investigation and experimental design through implementation and use in the product.

The work combines careful reasoning, the ability to dive into domain specifics, and software design. You'll design agent environments and how agents represent problems, gather evidence, use tools, retain context, revise their conclusions, and coordinate with each other. You'll also make the engineering decisions that let us inspect those processes, test changes, and build on what works.

Most of the initial work will involve systems built around existing frontier models. Training or fine tuning may be useful for particular problems, but the direction of the work will follow what helps users reason better.

• Understand the decisions we're helping people make. Work through real cases with users and domain experts. Identify their objectives, constraints, available evidence, and important unknowns. Learn enough about an unfamiliar problem to recognize when an answer misses the point.

• Define and measure good reasoning. Build evaluations for problems where success is initially under specified. Combine close inspection of examples, expert judgment, and quantitative analysis. Investigate whether a better score reflects a meaningful improvement, and whether that improvement holds on new cases.

• Improve the system's behavior. Turn observed failures into hypotheses you can test. Experiment with problem decomposition, context, tools, and ways for agents to seek information or check their work. Follow promising results through to changes users can benefit from.

• Design the software that makes this work reliable. Choose useful abstractions, data structures, and component boundaries for reasoning workflows and evaluations. Make deliberate tradeoffs among simplicity, flexibility,…