Research Engineer, RL Engineering

About Anthropic

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems that are safe and beneficial for users and society. Our team consists of researchers, engineers, policy experts, and business leaders working together to build beneficial AI systems.

As an ML Systems Engineer on our Reinforcement Learning Engineering team, you will build, maintain, and improve the algorithms and infrastructure that our researchers use to train models. You will focus on improving the performance, robustness, and usability of these systems, allowing research to progress quickly.

• Build, maintain, and improve algorithms and systems for training models.
• Enhance speed, reliability, and ease-of-use of training pipelines.
• Profile reinforcement learning pipelines for improvement opportunities.
• Build systems that launch training jobs in test environments to detect problems quickly.
• Adapt fine tuning systems for new model architectures.
• Instrument training code to detect and eliminate Python GIL contention.
• Diagnose and fix performance regressions in training runs.
• Implement stable, fast versions of training algorithms proposed by researchers.

• 4+ years of software engineering experience.
• Experience building systems and tools that increase others’ productivity.
• Results-oriented with a bias towards flexibility and impact.
• Willingness to pick up tasks outside your primary description.
• Enjoys pair programming.
• Interest in machine learning research.
• Concern for the societal impacts of your work.

• High performance, large-scale distributed systems.
• Large-scale LLM training.
• Python.
• Implementing LLM fine tuning algorithms, such as RLHF.

• Profiling our reinforcement learning pipeline to find opportunities for improvement.
• Building a system that regularly launches training jobs in a test environment to detect issues quickly.
• Making changes to our fine tuning systems so they work on new model architectures.
• Building instrumentation to detect and eliminate Python GIL contention in our training code.
• Diagnosing why training runs have slowed after some number of steps and fixing it.
• Implementing a stable, fast version of a new training algorithm proposed by a researcher.

$500,000- $850,000 USD.

Minimum education:

Bachelor’s degree or equivalent combination of education, training, and/or experience.
Minimum field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience.
Minimum years of experience:
Local job level requirements.

Location:

Staff are expected to be in one of our offices at least 25% of the time; some roles may require more office presence.
Visa sponsorship:
We sponsor visas, but not all roles; we will make reasonable efforts if you receive an offer.