Senior Software Engineer in Test

Location: US, Remote;
Poland, Ukraine

Title: Senior Software Developer in Test (SDET)

Department: Technology

Founded on the principle of democratizing access—first to music in 1999, now to creative expertise in 2025—
Napster has consistently been at the forefront of transformational technology shifts that expand markets and empower users. The company’s latest platform turns passive consumers into active creators, providing the interface layer between foundation AI models and human creativity. For more information, visit

Napster is looking for a Senior Software Developer in Test to own test infrastructure strategy for our real-time AI agent platform. This is not a QA-as-afterthought role. You'll be embedded in the engineering organization, making framework decisions that outlast any single sprint, leading the team toward scalable automation patterns across both deterministic and AI-driven systems, and building load testing tooling that gives us confidence before production sees a spike - with scope that grows beyond this product over time.

Test Framework Architecture

• Evaluate, propose, and drive adoption of test frameworks across the stack - including tooling for real-time, stateful session testing alongside conventional API and UI frameworks - with documented rationale, not just convention
• Define project structure, configuration standards, and execution model across multiple test layers: unit, contract, real-time/protocol, AI behavioral, and end-to-end
• Establish patterns for test data management, environment configuration, and cross-service dependency handling
• Champion a test pyramid strategy suited to a system that combines standard request/response APIs with persistent, real-time sessions

Automation Leadership

• Lead automation design reviews - define what gets automated, at what layer, and why
• Establish and document automation patterns: service abstractions, fixture strategies, retry logic, test isolation contracts, and behavioral baselines for systems with non-deterministic output
• Mentor engineers on writing testable code and maintainable specs; conduct test code reviews with the same rigor as production code reviews
• Drive shift-left practices: bring testing into the design phase, not just the PR stage

Load & Performance Testing Tooling

• Design and build internal load testing tooling and harnesses (extending k6, Locust, or Gatling) tailored to concurrent, real-time session load - not just standard HTTP throughput
• Develop reusable libraries for traffic shaping, scenario composition, and result aggregation
• Integrate load tests into CI/CD with configurable thresholds and automated regression alerts
• Build dashboards and reporting pipelines that translate raw load test output into actionable signals for engineering and product

AI System Quality & Real-Time Reliability

• Build evaluation harnesses for systems with non-deterministic, AI-generated output — defining what "passing" means in terms of statistical baselines and acceptable variance, not exact-match assertions
• Own test coverage for real-time, stateful protocols, including session lifecycle, data stream integrity, and timing-sensitive interaction handling
• Partner with engineering to identify where AI output quality and system reliability intersect, and design tests that catch regressions in both

Platform & Infrastructure

• Own test infrastructure: parallel execution environments, CI integration, flakiness detection, and test result observability
• Partner with Dev Ops/Platform teams to ensure test environments are production-representative
• Identify and remediate sources of test flakiness systematically

Required

• 4+ years of software engineering experience, with at least 2 years focused on test infrastructure, tooling, or automation engineering
• Strong proficiency in Type Script/JavaScript; additional experience with Python or Go a plus
• Demonstrated experience proposing and implementing a test framework from scratch or significantly evolving an existing one
• Hands-on experience with load testing tools (k6, Locust, Gatling, JMeter, or equivalent) and the judgment to extend them
• Experience testing real-time, streaming,…