Engineering Manager, AI

• Lead and grow an engineering team building and hardening Permit Flow's agentic infrastructure
• Build engineering culture through clear expectations, 1:1s, feedback loops, and career development
• Own technical architecture for workflow agents, including orchestration, model serving, latency, cost per task, and failure handling
• Partner with the Product Manager, AI to define reliability standards and release gates
• Build and scale evaluation infrastructure, including golden data sets, LLM-as-judge pipelines, human annotation tooling, and online monitoring
• Drive internal tooling for testing, error analysis, and hill-climbing agentic work
• Improve reliability, observability, and engineering efficiency across the AI stack
• Hire and develop engineers with ML/AI systems depth and future team leaders
• Own the roadmap from technical direction and execution planning through delivery and measurable production accuracy and reliability gains
• Report directly to the VP of Engineering and partner closely with the Product Manager, AI

Requirements

• 5+ years in software engineering, including 1+ year in engineering leadership
• Direct experience building or shipping a production AI/agentic system and owning its infrastructure
• Hands-on familiarity with evaluation infrastructure: golden data sets, LLM-as-judge, human annotation, or online monitoring
• Strong system design and architecture fundamentals
• Ability to lead through ambiguity and turn unclear, fast-moving problems into structured execution
• Deep technical fluency in code review, architecture review, and model/system tradeoffs
• Strong communication skills across technical and non-technical stakeholders, especially product
• High standards for engineering quality, reliability, and performance
• Bonus: experience in workflow-heavy, enterprise SaaS, or infrastructure products
• Bonus: exposure to complex regulated environments
• Willingness to work a hybrid schedule with 3 in-office days per week (MWF) in New York City
• Authorization to work in the United States or ability to address U.S. sponsorship requirements

Core Competencies

Demonstrates expertise in leading engineering teams and building AI/agentic systems, with a strong focus on technical architecture, reliability standards, and engineering efficiency. Proven ability to develop talent and drive high-quality engineering practices in a fast-paced environment.

Highest-signal resume keywords

• Engineering Leadership
• AI/Agentic System Infrastructure
• Evaluation Infrastructure Development
• System Design and Architecture
• Strong Communication Skills

Hard Skills

• Software Engineering
• Technical Architecture
• Model Serving
• Error Analysis
• Code Review
• Architecture Review
• Reliability Standards
• Performance Optimization
• Production Accuracy
• Latency Management

Soft Skills

• Team Development
• Feedback Loops
• Structured Execution
• Collaboration
• Problem-Solving

Industry Keywords

• Enterprise SaaS
• Workflow-Heavy Products
• Regulated Environments

Tools & Technologies

• Golden Data Sets
• LLM-as-Judge Pipelines
• Human Annotation Tooling
• Online Monitoring
• Internal Tooling