Sr. ML Infrastructure Engineer - Kleiner Perkins backed Stealth Startup

This range is provided by Day One Partners. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

$/yr - $/yr

Day One Partners is working with the founders of a stealth AI company backed by Tier 1 VCs. They’re building an AI-powered platform that helps biopharma companies bring new drugs to market faster and more cost-effectively. The company already has paying customers and strong interest from major industry players.

We’re hiring a Machine Learning Infrastructure Engineer to lead the design and development of their LLM evaluation systems. This role is onsite in South San Francisco and works closely with both the founding team and their first customers.

Why This Role Matters:

Developing and launching new drugs is expensive and slow—over $300M per drug, with high failure rates. This team is solving that by centralizing industry data and automating complex workflows. Your work will directly impact the accuracy, quality, and reliability of the data systems that support these critical decisions.

What You’ll Do:

• Design and build robust LLM evaluation frameworks from scratch
• Develop methods to measure LLM performance, identify errors, and ensure high data quality
• Own the infrastructure for extracting structured data from large volumes of unstructured documents
• Run experiments to improve data retrieval and extraction accuracy
• Collaborate directly with customers to understand data needs and explain evaluation methodologies
• Work with the founding team to shape the future of their AI-powered platform

What You Bring:

• 4+ years of experience in machine learning infrastructure, with a focus on building ML or LLM evaluation systems
• Proven experience designing and implementing evaluation frameworks for LLMs
• Strong software engineering skills
• Familiarity with classical ML methods (logistic regression, random forest, XGBoost)
• Excitement about working with large datasets and uncovering new signals
• Interest in life sciences, healthcare data, or similar domains is a plus
• Comfortable working onsite in a fast-paced, early-stage startup

Bonus Points For:

• Experience fine-tuning LLMs or deep learning models
• Previous startup experience, especially in high-growth companies
• Background in biology, biotech, or healthcare ML applications

If you have a strong track record building LLM evaluation systems and want to join a small team solving big problems, reach out to Day One Partners to learn more. For immediate consideration, email resumes to stephanie with the subject Day One ML Infra

Mid-Senior level

Full-time

Industries:
Biotechnology Research, Technology, Information and Media, and Software Development