Software Test Engineer

Residency

All applicants must currently reside in the United States

Ask Sage, a Big Bear.ai company
, is hiring a Software Test Engineer in to strengthen automated validation across our AI platform. We have a growing backlog of backend and platform work that requires deeper regression coverage, stronger system-level validation, and more scalable test execution. This is not a manual QA role, and it is not a Dev Sec role. We are looking for a software engineer who specializes in quality engineering: someone who can understand backend code, reason through distributed system behavior, write high-quality automated tests, and improve the infrastructure that gives engineers reliable release signal.

You will partner with engineers to define acceptance tests that prove a fix works and fail reliably if the fix is reverted, and your work will directly improve release velocity, platform reliability, and confidence in complex AI workflows.

• Develop and maintain smoke tests, unit tests, integration tests, and end-to-end tests across backend and platform workflows.
• Validate backend changes for new PRs through automated test coverage, direct API testing, and user-facing workflow verification where appropriate.
• Exercise backend behavior through the UI when useful, without owning visual design, frontend UX validation, or manual UI regression testing.
• Build regression coverage for edge cases, malformed inputs, authorization boundaries, concurrency issues, failure modes, and other non-happy-path scenarios.
• Improve the scalability, determinism, execution performance, maintainability, and diagnostic quality of the test suite.
• Strengthen test fixtures, mocks, test data management, failure analysis, and CI feedback loops.
• Design end-to-end tests for AI platform workflows while minimizing unnecessary token usage, external provider calls, latency, and test cost.
• Apply AI-assisted development and analysis tools to accelerate test design, test generation, triage, and maintenance while preserving reliability, reviewability, performance, and deterministic validation standards.
• Validate backend APIs, authentication flows, billing and token behavior, model routing, AI workflow execution, file parsing, MCP/tool execution, and passthrough APIs.
• Validate AI platform E2E paths involving prompts, model responses, streaming behavior, tool calls, agents, workflow orchestration, and provider-facing API compatibility.
• Validate agentic harnesses, MCP integrations, and workflow automation systems, including concepts common to no-code and low-code workflow builders such as Power Automate, Zapier, Make, n8n, and similar platforms.
• Validate security-sensitive product behavior such as user isolation, permission boundaries, validation, sanitization, rate limits, replay prevention, safe error handling, and layered control behavior.
• Build and maintain test infrastructure that must scale with a growing platform, expanding product surface area, and active engineering team.

• 8-10 years of experience with software testing
• Strong software engineering background with deep experience in automated test development.
• Experience testing backend services, APIs, distributed systems, and database-backed applications.
• Experience designing smoke, unit, integration, and end-to-end testing strategies.
• Strong judgment around edge cases, non-happy paths, adversarial inputs, regression risk, and failure isolation.
• Experience improving CI test reliability, execution time, parallelization, test isolation, and failure observability.
• Familiarity with testing Defense in Depth behavior: validating that multiple layers of checks work together, without this being a dedicated Dev Sec role.
• Strong troubleshooting, analytical, and communication skills.
• Ability to work independently and as part of a team.
• Ability to obtain clearance.

• Experience writing E2E tests for AI platforms, LLM applications, model gateways, agents, MCP tools, or tool-using systems.
• Experience designing AI E2E tests that control token usage, external provider calls, latency, and cost.
• Experience testing workflow automation systems, agentic harnesses, or no-code/low-code automation platforms (e.g., Power Automate, Zapier, Make, n8n).
• Familiarity with flagship Generative AI provider APIs (Google VertexAI, AWS Bedrock, Microsoft Azure OpenAI) and models (OpenAI GPT, Anthropic Claude, Google Gemini).
• Experience with CI/CD pipelines (e.g., Git Hub Actions), Docker, Kubernetes, and…