Principal Engineer, Behavior Understanding and Evaluation

Mission Summary

Mission

Summary:

The Behavior Understanding and Evaluation team at Motional is dedicated to comprehending and assessing the intricate behavior of the Motional Vehicle. To prepare for unprecedented scale, it's paramount that we leverage automated algorithms to thoroughly understand and evaluate our vehicles' performance, both in real-world deployment and within simulated environments.

We aim to employ cutting-edge machine learning techniques in our evaluation processes, ensuring we can scale effectively to meet Motional's ambitious goals. Our collaborative approach extends across various teams, providing critical insights and actionable metrics to those developing the onboard software and enhancing the rider experience. We are responsible for building and operating robust, large-scale machine learning and data systems, alongside sophisticated simulation workflows and insightful diagnostic tools.

By combining expert human judgment with advanced machine learning models, we generate essential training and evaluation data for the Motional driver.

We are seeking passionate researchers and software engineers to join us in building production-grade machine learning evaluation systems for autonomous vehicles - people who are driven to transform complex data into measurable progress toward a safer, smarter transportation future.

• Define the strategic direction for the development and application of machine learning-driven evaluation methodologies for autonomous driving.
• Optimize large-scale evaluation infrastructure to ensure scalability.
• Partner cross-functionally with Behaviors, Actions, Simulation, System Engineering, Research, and other Onboard/Offboard Software Teams on evaluation strategies.
• Upgrade evaluation pipelines by enhancing existing Heuristic and Event based processes with learned methods
• Uphold scientific and statistical rigor in the application of embodied AI.
• Pioneer the definition and implementation of key evaluation metrics and reward functions, driving autonomy performance across safety, comfort, and assertiveness.
• Align stakeholders around clear evaluation objectives and priorities.

• Proven experience in establishing and executing evaluation strategies.
• A strong foundation in scientific and statistical methodologies.
• Expertise in Machine Learning and Deep Learning.
• At least 3 years of experience leading and scaling large-scale data and evaluation systems.
• Proficiency in crafting data-driven reports and technical strategy documents for executive decision-making.
• Technical leadership experience, including hands-on involvement in the development of large-scale AI or VLM/LLM systems.

• Experience in mentoring and leadership development.
• A background in shaping product roadmaps.
• Familiarity with Reinforcement Learning (RL) with Human Feedback.
• Specialized expertise in creating expansive datasets.
• Experience with large-scale global consumer products.

We encourage a hybrid schedule with in-office time at one of our locations in Boston, Pittsburgh, or Las Vegas to support collaboration, or this role can be fully remote.

The salary range for this role is an estimate based on a wide range of compensation factors including but not limited to specific skills, experience and expertise, role location, certifications, licenses, and business needs. The estimated compensation range listed in this job posting reflects base salary only. This role may include additional forms of compensation such as a bonus or company equity.

The recruiter assigned to this role can share more information about the specific compensation and benefit details associated with this role during the hiring process.

Candidates for certain positions are eligible to participate in Motional’s benefits program. Motional’s benefits include but are not limited to medical, dental, vision, 401k with a company match, health saving accounts, life insurance, pet insurance, and more.

Salary Range $175,000 — $234,250 USD

Motional is a driverless technology company making autonomous vehicles a safe, reliable, and accessible reality. We’re driven by something more.

Our journey is always people…