Skip to content
×

Member of Technical Staff, Agents

Listing for: Royper Royco LLC
Full Time position Listed on 2026-11-19
Job specializations:
Salary/Wage Range or Industry Benchmark: 200000 - 300000 USD Yearly USD 200000.00 300000.00 YEAR

Job Description & How to Apply Below

Join a small, senior team building autonomous AI agents for executive workflows. You will develop agent capabilities and evaluation systems that help ensure real-world tasks are completed reliably, with measurable improvements reaching users quickly.

What You'll Do
  • Build agents that complete delegated work end to end, without requiring users to check each step.
  • Define measurable success criteria and trusted ways to assess agent performance across tasks.
  • Develop and evaluate new agent capabilities before releasing them to users.
  • Investigate production failures and deliver durable fixes based on their underlying causes.
  • Build systems to detect known failure patterns and less predictable issues at scale, using tools such as Braintrust and Raindrop.
  • Improve agent recovery when tools or websites fail.
  • Create repeatable evaluation workflows that let engineers measure whether changes improve agent performance.
What We're Looking For
  • At least 3 years of data science or machine learning experience building evaluation systems, metrics, or product quality measurement.
  • Production-quality Python and SQL skills for analysis, large datasets, and automated pipelines.
  • Experience designing evaluation frameworks, representative datasets, meaningful metrics, and realistic end-to-end simulations.
  • A track record delivering production AI, ML, agent, or agent-evaluation systems for real users and measuring their impact.
  • Experience improving agent reliability, owning systems end to end, and using statistics and experiment design to evaluate changes.
  • Familiarity with post-training, fine-tuning, or reinforcement learning is useful.
  • Experience with Django, React, and Type Script; the role also uses Python, SQL, Braintrust, and Raindrop.
  • Demonstrated progression and sustained experience in a role, with comfort working in a fast-moving, ambiguous environment.

Base salary: $200,000 to $300,000 annually. Visa sponsorship is not available.

To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary