Data Scientist: AI Evaluation & Context Systems

Staff Data Scientist: AI Evaluation & Context Systems

Join Demand.io as a Staff Data Scientist and be the lead architect for our Evaluation & Context Systems. This high‑leverage role is for a builder who wants to define the mathematical standard of "Truth" for autonomous commerce agents.

Demand.io is a profitable, founder‑led consumer AI commerce player driving $1B+ in annual GMV and behind products like Simply Codes and Product.ai. We are building a deterministic AI that eliminates hallucinations in commerce.

• The Evaluation Architecture (Benchmarking Truth): Design a proprietary evaluation harness, golden sets, and adversarial loops that benchmark our AI’s ability to distinguish marketing claims from verified facts. Define metrics that prevent hallucinations from ship.
• Neuro‑Symbolic Retrieval (Graph
  
  RAG): Architect a retrieval system that fuses our Commerce Knowledge Graph with vector search so the agent retrieves the logic of an entity, not just semantic similarities.
• The Economics of Intelligence: Own the unit economics of intelligence, modeling the trade‑offs between neural reading and symbolic logic and optimizing the stack for risk‑adjusted value per token.

• You are a Scientific Engineer: You design rigorous protocols, understand statistical significance, and guard against p‑hacking.
• You obsess over "The Tails": You dig into distribution tails to understand why the 10% failed.
• You are an Anti‑Academic: You ship systems for 10 million users, not papers.
• You treat Evaluation as Code: You automate test case generation rather than writing thousands of unit tests.

• Base Salary: $250,000 – $380,000 per year.
• Profit Participation (PIUs): Grants of Profits Interest Units that give you true ownership and cash‑flow distributions tied to company profits.
• Benefits: 100% premium coverage for you and your family, daily catered lunches, and unlimited PTO that is truly expected to be used.

• Annual bonus and stock options.

• The Input: Answer five short questions via async video instead of a phone screen.
• The Context: High‑signal prompts designed to test prioritization and complexity compression; rehearsal not required.
• The Trade: Clear signals skip the generic recruiter screen; you move straight to a peer‑level conversation.

We don't use cover letters or ATS keyword bots. Submit your resume via the link provided and let your logic speak.