QA Engineer, Voice AI Agents

Location: United States

Workplace: Remote

Employment Type: Full-Time

Experience: 3-5 years in QA, support engineering, technical operations, or a related role

Core Areas: Software Quality Assurance, Voice AI Agents, Production Debugging, Conversation QA, Root Cause Analysis, Regression Testing, SQL, Test Automation

Compensation: $180,000-$200,000 annually

This opportunity is for a QA Engineer, Voice AI Agents responsible for investigating production issues, validating conversational AI workflows, and improving software quality across live voice AI campaigns. The role focuses on identifying defects, analyzing call behavior, tracing technical failures, and ensuring reliable agent responses, scheduling outcomes, escalation paths, and system integrations. Responsibilities include hands-on debugging, pre-launch testing, ongoing quality monitoring, and development of structured QA processes.

Reporting to the Head of Product, this position works with software engineers, deployment managers, and clinical and nurse operations teams to resolve platform issues and prevent recurring failures. The work involves analyzing call recordings, conversation transcripts, system logs, campaign configurations, and integration data, while using SQL, APIs, JSON or YAML, and automation techniques to improve testing efficiency and production reliability.

• Investigate reported and proactively detected production issues across live voice AI campaigns, including dropped calls, incorrect agent responses, failed handoffs, scheduling errors, and data mismatches.
• Analyze call recordings, conversation transcripts, system logs, campaign configurations, and integration data to identify the underlying causes of technical and conversational failures.
• Perform root cause analysis to distinguish between AI agent behavior, configuration errors, data inconsistencies, integration problems, and software platform defects.
• Document defects through detailed, reproducible bug reports that include severity, affected members, business impact, supporting evidence, and actionable information for engineering teams.
• Triage incoming production issues, assess severity and priority, and route defects to the appropriate engineering, deployment, or operational owners.
• Coordinate with engineers, deployment managers, and clinical and nurse operations staff to investigate issues, support resolution, and improve system reliability.

• Execute pre-launch quality assurance testing for new campaigns, workflow updates, and configuration changes before production deployment.
• Validate test scripts, workflow scenarios, edge cases, escalation procedures, and handoffs to nurses or live support staff.
• Conduct ongoing conversation QA for active campaigns, evaluating agent interactions against established quality standards and identifying recurring performance issues.
• Verify campaign targeting, sequencing, disposition logic, and outcome capture to ensure workflows operate according to defined requirements.
• Identify quality trends, emerging defects, and potential operational risks before they develop into customer-facing issues.

• Develop and maintain structured QA processes, including test cases, quality scoring rubrics, regression test suites, and release readiness checklists.
• Identify opportunities to automate repetitive quality checks, including transcript analysis, campaign configuration validation, and outcome reconciliation.
• Support the implementation and improvement of automated QA checks to increase testing consistency and reduce manual investigation effort.
• Track quality metrics, analyze recurring defects, and communicate findings to product, engineering, and deployment teams.
• Use quality trends and investigation findings to recommend improvements that address the root causes of production issues.

Required Experience

• 3-5 years of professional experience in software quality assurance, support engineering, technical operations, or a closely related role supporting a production software platform.
• Experience creating and executing test plans and documenting reproducible software defects through clear, actionable bug reports.

Required Skills

• Strong technical debugging and root cause analysis skills, including the ability to investigate complex production issues and validate findings using supporting evidence.
• Ability to read and interpret system logs, query…