Member of Technical Staff - QA Engineer

Overview

Patronus AI is a frontier lab developing simulation research and infrastructure to accelerate progress toward human-aligned AGI. We are on a mission to simulate all of the world’s intelligence.

We are the team behind some of the earliest and most influential research in AI evaluation like Finance Bench, Lynx, Simple Safety Tests , Copyright Catcher, Humanity’s Last Exam, and more. We are formerly AI researchers and engineers from companies like Meta AI, Amazon AGI, and Google. Our customers include foundation model labs and Fortune 500 enterprises like Adobe. We are backed by top-tier investors like Lightspeed Venture Partners, Notable Capital, Stanford University, Noam Brown, Gokul Rajaram, and more.

You are responsible for ensuring the quality and reliability of our products and systems. You will work closely with our engineering, AI and design teams to identify and resolve any potential issues or bugs, create testing plans and datasets, ensuring that our products meet the highest standards of performance and functionality. Successful applicants are skilled, have prior QA experience, and are extremely detail-oriented.

• Develop and execute test plans to ensure the quality of software features. Create high quality, specific test cases that cover diverse scenarios and edge cases to catch unexpected behavior.
• Ensure that production releases are well tested. Typically releases happen on a weekly cadence; QA will ensure reliability of new features in the web platform, collaborating closely with frontend engineers
• Regularly report bugs and issues from testing results and file tasks;
  Triage to engineers accordingly
• Collaborate with the engineering and AI teams to understand requirements for testing;
  Participate in technical design meetings as needed
• Construct datasets for AI research, working with the AI team, including datasets for model training, fine-tuning and evaluation
• Develop and maintain automated testing scripts for our API
• Help customer success team investigate and troubleshoot customer-reported issues to identify root causes and recommend solutions
• Executing load tests
• Run benchmarking scripts to assess evaluator performance from configuration changes, e.g. a change in the model selection, underlying dataset or other parameters

Above all, we look for an eagerness to learn, passion for research, creativity in problem solving and a proactive mindset. You are a great fit if you have a background in the following:

• 3+ years of experience as a QA engineer working on web platforms in a production environment
• Experience with REST APIs
• Experience performing manual application tests
• Experience writing end-to-end tests
• Experience with Playwright, Selenium, Cypress
• Experience with tools like Postman, Swagger, OpenAPI
• Have good character, integrity, and respect for others!
• Competitive salary and equity packages
• Health, dental, and vision insurance plans
• 401(k) plan + matching
• Sponsored personal tax accounting
• Whoop band, Oura ring, Function Health
• Monthly meal stipend
• Monthly health and wellness stipend
• Fun global offsites!

Patronus AI is an equal opportunity employer. We celebrate diversity in our workplace, and all qualified applicants will receive consideration for employment without regard to age, ancestry, color, family or medical care leave, gender identity or expression, genetic information, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran status, race, religion, sex (including pregnancy), sexual orientation, or other legally protected characteristics.