Systems Engineer - Behavior Verification

• Design comprehensive verification strategies:
  Develop, structure, and document scenario-based test sets that assess autonomous driving behavior, ranging from minimum release requirements to comprehensive performance evaluations.
• Govern software release readiness:
  Define and execute scenario validity checks, review test set quality, and approve or reject test sets before they are used to evaluate software releases.
• Maintain test data quality:
  Establish continuous quality feedback processes for scenarios, taggers, graders, and the taxonomies supporting behavior verification, ensuring that test artifacts remain accurate, consistent, and trustworthy.
• Translate requirements into measurable criteria:
  Collaborate with Behavior Requirements teams to convert behavioral expectations into clear, testable acceptance criteria, performance targets, and formally agreed verification standards.
• Investigate behavioral deficiencies:
  Analyze identified failures, search for additional instances, quantify the system's exposure to relevant scenarios, and feed missed issues back into the verification pipeline as test escapes.
• Distinguish software failures from simulation limitations:
  Determine whether observed behavioral issues represent genuine system deficiencies or simulation artifacts, and communicate simulation fidelity gaps to the appropriate engineering teams.
• Coordinate cross-functional verification activities:
  Work closely with motion planning, simulation, evaluation, operations, safety methodology, and platform teams to align interfaces, responsibilities, and verification processes.
• Define verification phases and cadence:
  Establish appropriate testing frequencies and verification requirements for daily software releases, significant exposure increases, and other critical development milestones.
• Strengthen release decision-making:
  Provide clear, evidence-based assessments of autonomous driving performance to support decisions about software readiness, risk, and release progression.
• Improve verification processes:
  Identify gaps in test coverage, refine test strategies, and contribute to the development of scalable verification methodologies as the team and technology evolve.

• Bachelor's or master's degree in a relevant engineering or technical discipline, such as Software Engineering, Electrical Engineering, Computer Science, Robotics, Advanced Controls, Aerospace Engineering, or a related field.
• Professional experience verifying safety-critical systems, ideally within autonomous vehicles, advanced driver‑assistance systems, robotics, or other complex engineering environments.
• Demonstrated experience designing test cases from system requirements and evaluating test coverage, including identifying gaps and limitations in existing test sets.
• Applied statistical knowledge, including sample size selection, confidence assessment, and the interpretation of test results and their limitations.
• Proficiency in Python and the ability to navigate, understand, and contribute to a large and complex codebase.
• Familiarity with autonomous vehicle requirements, driving behaviors, and systems engineering or design methodologies.
• Strong organizational skills and the ability to structure, document, prioritize, and maintain large collections of test sets and verification strategies.
• Excellent analytical and problem-solving abilities, with a rigorous approach to evaluating evidence, identifying failures, and distinguishing genuine behavioral issues from testing or simulation artifacts.
• Strong cross-functional collaboration and communication skills, including the ability to work diplomatically with teams that may have differing priorities or technical perspectives.
• A high degree of ownership, accountability, and commitment to quality, with the confidence to make evidence-based decisions and uphold verification standards.
• Adaptability and a proactive mindset, with the ability to establish processes, resolve ambiguity, and contribute effectively in a developing technical organization.

Preferred qualifications:

• Experience with trucking, heavy vehicles, commercial transportation, or other large-scale autonomous vehicle applications.
• Knowledge of machine learning systems, particularly model evaluation and the trade-offs between false positives and false negatives.
• Familiarity with relevant safety and regulatory frameworks, including NHTSA guidance, ISO 26262, and Safety of the Intended Functionality (SOTIF).
• Experience using tools and methodologies to…