Senior VLM Research Lead, Autonomy

Rivian Tech Lead Role

Rivian is on a mission to keep the world adventurous forever. This goes for the emissions-free Electric Adventure Vehicles we build, and the curious, courageous souls we seek to attract. As a company, we constantly challenge what's possible, never simply accepting what has always been done. We reframe old problems, seek new solutions and operate comfortably in areas that are unknown.

Our backgrounds are diverse, but our team shares a love of the outdoors and a desire to protect it for future generations.

Role

Summary:

Vision-Language Models (VLMs) are a foundational pillar of our Autonomy stack. In this Tech Lead role, you will drive and deliver the overarching VLM strategy, which includes training and shipping VLM models, extending to multi-modalities, enabling new use cases, among others. In this role, you will also be responsible to architect VLM-driven solutions to solve some of autonomy's hardest challenges, including automated data mining, handling long-tail distributions, rare edge-case detection, and scene anomaly reasoning.

You will also drive our large-scale training data acquisition strategy for VLM-related model training, closely collaborating with our teams and partners. You will also own the whole end-to-end lifecycle of VLM model delivery: data acquisition, metrics definition, benchmarking, model performance optimization, deployment, feedback loop. Collaborating broadly across the Autonomy org, you will serve as the champion for VLM models and data mining capabilities, as well as represent these efforts in our interactions with other teams.

Responsibilities:

• Drive and deliver the VLM strategy:
  Own the holistic roadmap of the VLM strategy, including training and delivering VLM models, deployment, alignment, and ensuring a unified vision across the Autonomy org.
• Accelerate data mining:
  Design and deliver VLM-related models and strategies that power automated data mining, long-tail distributions, rare/edge case detection, and anomaly detection at scale.
• Drive and deliver the data acquisition strategy:
  Architect the strategy for large-scale training data acquisition to train the VLM models and improve their performance, establishing workflows with in-house and 3rd-party annotation vendors.
• Iterate and optimize performance:
  Establish rigorous evaluation and monitoring benchmarks. Identify and root-cause top-tier system anomalies, prioritizing high-impact optimizations to continuously push the needle on performance.
• Cross-functional collaboration:
  
  Partner closely with core Autonomy teams (Perception, Planning, Calibration, Systems, etc) to translate vehicle feature requirements into concrete ML deliverables.

Qualifications:

• Education:
  
  BS, MS, or PhD in Computer Science, Robotics, Electrical Engineering, or a highly related quantitative field.
• Experience:
  
  5+ years of professional experience scaling ML solutions, with a strong focus on the following:
• VLM model training:
  Hands-on experience training or fine-tuning VLMs using modern parameter-efficient techniques (LoRA, QLoRA) and RL alignment.
• Large-scale data mining:
  Proven track record developing VLM/LLM-related techniques for data mining, long-tail distributions, rare cases, safety-critical events.
• Zero/few-shot capabilities:
  Experience with open-vocabulary, zero-shot, or few-shot classification models, particularly in long-tail scenarios.
• Training data strategy:
  Experience with driving training data acquisition strategy to train VLM-related models, defining data annotation guidelines, partnering effectively with in-house and external 3P annotation vendors.
• System engineering:
  Strong proficiency in Python alongside a solid understanding of modern Perception pipelines, benchmarking tools, and infrastructure.
• Execution:
  Demonstrated ability to root-cause complex issues across a distributed, cross-functional stack in a fast-paced environment.

Pay Disclosure:
The salary range for this role is $,300 for San Francisco Bay Area based applicants. This is the lowest to highest salary we in good faith believe we would pay for this role at the time of this posting. An employee's position within the salary range will be based on several factors including, but not limited to, specific competencies, relevant education, qualifications, certifications, experience, skills, geographic location, shift, and organizational needs.

We offer a comprehensive package of benefits for full-time and part-time employees, their spouse or domestic partner, and children up to age 26, including but not limited to paid vacation, paid sick leave, and a competitive portfolio of insurance benefits including life, medical, dental, vision, short-term disability insurance, and long-term disability insurance to eligible employees. You may also have the opportunity to participate in Rivian's 401(k) Plan and Employee Stock Purchase Program if you meet certain eligibility requirements.

Full-time employee coverage is effective on their first day of employment. Part-time employee coverage is…