Senior Software Engineer - Agents

At Qualtrics, we create software the world’s best brands use to deliver exceptional frontline experiences, build high-performing teams, and design products people love. But we are more than a platform— we are the creators and stewards of the Experience Management category serving over 18K clients globally. Building a category takes grit, determination, and a disdain for convention—but most of all it requires close-knit, high-functioning teams with an unwavering dedication to serving our customers.

When you join one of our teams, you’ll be part of a nimble group that’s empowered to set aggressive goals and move fast to achieve them. Strategic risks are encouraged and complex problems are solved together, by passing the mic and iterating until the best solution comes to light. You won’t have to look to find growth opportunities—ready or not, they’ll find you.

From retail to government to healthcare, we’re on a mission to bring humanity, connection, and empathy back to business. Join over 5,000 people across the globe who think that’s work worth doing.

Agent Studio is transforming how enterprises solve customer problems in real time. Your work on the testing and monitoring team directly enables customers to confidently deploy AI agents that reduce support escalations. We need a senior engineer who can build the observability and confidence systems that let our customers measure success and iterate rapidly. You'll work closely with product, science, and customer teams to ensure every agent deployed meets quality standards and delivers measurable business impact.

• Takes ownership of features from day one, learning new domains quickly and driving them to completion without needing task-by-task direction
• Writes effective prompts and debugs LLM graphs using tools like Lang Smith, bridging traditional engineering with AI/ML work
• Collaborates across teams (admin, runtime, product, science) to understand system behavior and customer needs
• Uses AI tools like Claude and Cursor fluently to accelerate development and problem-solving
• Communicates clearly with both technical and non-technical stakeholders, translating monitoring insights into actionable customer guidance
• Demonstrates customer obsession by prioritizing metrics and insights that help customers succeed
• Balances full-stack capability, comfortable moving between backend systems and UI work as priorities shift

• Deepen expertise in AI/ML observability and monitoring systems as the product scales
• Lead increasingly complex initiatives as the team grows and takes on new responsibilities
• Develop product thinking by working directly with customers and understanding how monitoring drives business outcomes
• Expand full-stack skills across both backend infrastructure and customer-facing interfaces

• Build and evolve monitoring systems that give customers real-time visibility into agent performance, enabling them to measure success and identify improvement areas
• Design and implement test coverage frameworks that give customers confidence their agents are production-ready before deployment
• Partner with the science team to develop LLM evaluation metrics and debugging tools that help customers optimize agent behavior
• Collaborate with forward deployment engineers and customers to identify monitoring gaps and translate feedback into product improvements
• Contribute to the platform’s transition from incubation to general availability by ensuring monitoring and testing scale reliably across growing customer base
• Participate in on-call rotation and support customers in production, using real-world issues to drive product improvements

• 5+ years of software development experience with demonstrated ownership of features or systems from conception through production
• Experience building or working with monitoring, observability, or testing systems at scale
• Proven ability to work across full-stack domains (backend and frontend) in a fast-moving environment
• Track record of learning new technical domains quickly and shipping quality work
• Experience working with AI/ML…