Senior AI Engineer - San Mateo, CA

Location

San Mateo, California - USA

Full time

Hybrid

Engineering

Trust Lab deploys cutting edge solutions to evaluate AI agents, models and apps for enterprise customers. With a 5 year track record working with large and small clients including social media companies and digital market places, and guided by founders who previously worked in senior leadership positions at Google, You Tube, Tik Tok, and Reddit, we are creating industry leading LLM based solutions for agentic system evaluation and labeling.

Our approach includes human-in-the-loop and LLM-as-a-judge technologies, with a focus on rapid innovation and production level scaling. You’ll join a small, mission-driven team where your contributions have a direct impact on real-world issues.

What you’ll do: At Trust Lab, your work won’t live in theory - it will power live systems used at large scale. You’ll develop, tune, and optimize LLM-driven solutions that interpret and reason about complex digital content, while experimenting rapidly from design to deployment and seeing immediate feedback from real-world use cases. Partnering closely with other engineers, researchers, and product leaders, you’ll pioneer new approaches to model training and evaluation, taking ownership from early R&D through to production launches, and ensuring your work directly shapes how millions of people experience AI-powered content.

• Train, evaluate, and monitor new and improved LLMs and other algorithmic models
• Test and deploy content moderation models in production, and iterate based on real-world performance metrics and feedback loops.
• Develop medium to long-term vision for content understanding-related R&D, working with management, product, policy & operations, and engineering teams.
• Take ownership of results delivered to customers, pushing for change in approach where needed and taking the lead on cross-functional execution.

• Bachelor's or Master’s degree in Computer Science, Artificial Intelligence, Machine Learning, or a related field. Ph.D. is a plus. Proficiency in Python. Experience with AWS and CI/CD processes & tools is a strong plus.
• Experience with prompt-engineering techniques and familiarity with multiple LLM providers.
• Several years of industry experience in NLP / Computer Vision, or making LLM’s work in production for non-trivial use cases, incl. familiarity with evaluation metrics for classification tasks and best practices for handling imbalanced datasets.
• Hands-on experience with debugging issues in production environments, especially on AWS.
• Strong track record delivering results under time and resource pressure

• Work with a group of renown industry leaders in AI and Online Safety to shape the future of the industry.
• Ample opportunity and support for growth, as a technical individual contributor, or manager.
• Apply AI technology to real-world business use cases at a significant scale, with blue chip customers
• Work as part of a team where you can know everyone, but don’t have to do everyone’s job.
• Competitive compensation, comprehensive benefits, and hybrid in-office policy.