Safeguard Evaluation Software Engineer

Join Anthropic as a Software Engineer specializing in safeguard evaluations to improve AI systems' safety. Collaborate across ML research and engineering to develop robust evaluation metrics.

This role is pivotal in building the evaluation infrastructure for AI misuse detection. You will define metrics, construct datasets from real-world scenarios, and ensure systems evolve through rigorous testing. Your insights will directly influence safety protocols and enforcement actions in AI applications.

Key Responsibilities:

• Define metrics and test cases for investigative agents

• Create datasets from real traffic patterns

• Measure agent performance on detection and robustness

• Identify coverage gaps to refine evaluations

• Integrate successful methods into automated release pipelines

Requirements:

• Expertise in Python and data pipeline management

• Familiarity with LLM capabilities in context

• Proven data analysis skills for large datasets

• Experience transitioning experiments to production

• Six or more years in relevant software engineering

Drive innovation in AI safety practices and foster responsible technology at Anthropic.
#J-18808-Ljbffr