Senior Software Development Engineer in Test

At Qualia, we've built the leading B2B real estate technology that transforms the home buying and selling experience into a simple, secure, and enjoyable process. Our SMB and Enterprise products bring together users from across the real estate ecosystem—home buyers and sellers, lenders, title and escrow agents, and real estate agents—onto a single shared digital closing platform, providing greater clarity and transparency to real estate transactions.

Today, through our business customers across the country, millions of consumers use Qualia to close on homes every year.

WHAT YOU'LL WORK ONWe are hiring pioneering and collaborative engineers to build our next generation of intelligent products. Our quality engineers are conscientious hackers who thrive on collaboration and are adept at driving large-scale projects in the fast-changing domain of artificial intelligence. You will help build the automation, tooling, observability, and quality systems that let us ship complex, AI-driven products with confidence, bringing rigor and trust to a data-rich industry.

As a Senior AI/Agentic Quality Infrastructure SDET, you're a software engineer first. You build test infrastructure, agentic validation systems, AI/ML evaluation harnesses, observability platforms, and engineering tooling—not manual test cases or collections of automation scripts. You will help evolve our quality architecture from primarily deterministic automation toward a combination of traditional automation and agentic validation, building systems where AI agents can understand application state, execute workflows, use engineering tools, validate outcomes, investigate failures, and provide evidence for their conclusions.

You can take a hard, loosely defined problem and solve it. You're trusted to make technical decisions on frameworks, architecture, evaluation strategy, observability, and validation across every layer of the testing pyramid. You work closely with Engineering and Quality leadership, understand the systems you're testing at a deep technical level, and mentor engineers on building reliable tests, tooling, infrastructure, and AI-assisted validation systems.

RESPONSIBILITIES Build and maintain test infrastructure, AI/ML evaluation systems, agentic validation harnesses, release orchestration tooling, observability, and performance testing platforms as production-grade software rather than scripts

Design agentic test harnesses that let AI agents interact with applications, browsers, APIs, databases, logs, telemetry, and CI/CD systems to execute and validate complex workflows, reasoning about application state rather than relying solely on predefined test scripts

Build AI/ML evaluation frameworks for validating non-deterministic systems, including deterministic assertions, semantic and model-based evaluation, regression datasets, and human-in-the-loop review, defining measurable quality signals such as correctness, reliability, latency, cost, and hallucination/error rates

Build observability and telemetry into our automation and agentic infrastructure—structured logs, metrics, distributed tracing, and execution timelines—so engineers can understand why an automated or agentic workflow succeeded or failed

Develop systems that automatically distinguish product, test, infrastructure, and AI/model reasoning failures, correlating them with logs, traces, deployments, and commits to accelerate root-cause analysis

Design and build our test automation frameworks, including the migration from Cypress to Playwright, and own core infrastructure such as test environment provisioning, parallelization, and CI/CD integration at scale

Apply AI to engineering-quality problems including test generation, flaky-test triage, failure clustering, and…