Member of Technical Staff - Platform

Building LLM evaluation benchmarks and AI dev tools tied to vibe-coding benchmarking; directly involved in evaluating model behavior.

Join Vals AI as a platform engineer responsible for building and owning the scalable infrastructure that runs LLM evaluation benchmarks. The role spans distributed systems, cloud infrastructure, backend libraries, and full‑stack features (Python/Django backend and React/Type Script frontend) on an in‑person team based in San Francisco.

Member of Technical Staff - Platform responsible for owning and developing the platform that runs large-scale LLM benchmarks. You will build and maintain distributed systems, cloud infrastructure, internal and public libraries, and full-stack features to evaluate models reliably and efficiently.

• Build distributed systems to run evaluations across multiple models, benchmarks, and machines at scale.
• Deploy cloud infrastructure using Infrastructure as Code, including pipelines, servers, logging, and monitoring.
• Develop and maintain internal and external libraries, including the public model library.
• Implement full-stack platform features using React/Type Script on the frontend and Python/Django on the backend.
• Perform code and architecture reviews and help establish engineering best practices.
• Collaborate closely with research to ensure infrastructure meets evaluation needs.

• Strong engineering fundamentals with track record of building and shipping substantial projects.
• Significant professional experience with Python.
• System design experience and familiarity with VMs, containerization, load balancers, databases, etc.
• Familiarity with LLMs and experience working with LLM APIs and concepts like temperature and tokenization.

• Experience with development sprints, Git workflows, and pull request reviews.
• Strong verbal and written communication and feedback skills (e.g., design docs).
• Comfort with ambiguity and ability to decompose fuzzy problems.
• High iteration speed and an ownership mindset.

• Frontend development experience, ideally React.
• Experience with Django, FastAPI, or other Python HTTP servers.
• Experience with AWS infrastructure and IaC.
• Early-stage startup experience or having run your own company.
• Interest in AI/ML systems and evaluation.

Python, Django backend, React + Type Script frontend, AWS infrastructure using CDK for IaC.

This is an in-person role based in San Francisco. The company will support relocation or transportation as needed.

• Compensation range: $140,000 - $185,000.
• Relocation and transportation support; $1,500 housing stipend (within one-mile radius).
• Health and dental insurance coverage; 401K plan.
• Lunch and dinner provided, free snacks/coffee/drinks.
• Unlimited PTO; meaningful ownership/equity.

Python Django React Type Script AWS CDK FastAPI LLM APIs IaC

System Design Distributed Systems Cloud Infrastructure Infrastructure as Code Full-stack Development Python Code and Architecture Reviews Collaboration Communication Ownership Rapid Iteration Learning Velocity LLM Familiarity Problem Solving

Senior

USD 140,/year

Full-time

• Dental insurance coverage
• Lunch and dinner provided
• Free snacks/coffee/drinks
• $1,500 housing stipend (within one-mile radius)