Automation Engineer

Automation Engineer About Blitzy

Blitzy is a Cambridge, MA based AI software development platform on a mission to revolutionize the software development life cycle by autonomously building custom software to unlock the next industrial revolution. We're transforming how enterprises build software, turning enterprise requirements into production-ready code with an agentic software development platform that can autonomously execute 80% of the quantum of software development work.

We're backed by multiple tier 1 investors, and have proven success as founders of previous start-ups.

We are looking for an Automation Engineer to own, extend, and scale our automated testing frameworks that validate the runtime behavior of code generated by our AI platform. This role is critical in ensuring that our platform’s output performs correctly in production environments. You’ll work at the intersection of AI and quality assurance, expanding test coverage and improving the testing solutions that validate AI-generated applications during actual execution.

• You expand and maintain automated testing frameworks that validate the runtime behavior and performance of AI-generated applications.

• You develop intelligent test suites that automatically assess application functionality in production environments.

• You extend existing testing frameworks to increase coverage and validate runtime behavior across multiple languages and frameworks.

• You establish quality metrics and testing protocols that measure real-world performance of generated applications.

• You build systems to automatically detect and flag runtime issues in deployed applications.

• You collaborate with our AI team to improve the platform based on runtime performance data.

• You implement automated integration and end-to-end testing that ensures generated applications work as intended in production.

• You develop metrics and monitoring systems to track runtime performance across different customer deployments.

• Own the evolution and expansion of our automated testing frameworks for AI-generated code, increasing coverage and reliability.

• Drive and expand end-to-end and integration testing coverage across production environments.

• Drive runtime monitoring and quality assurance standards across multi-language deployments.

• Manage CI/CD pipeline integration for continuous quality validation.

• Collaborate cross-functionally with AI engineers to surface and resolve platform-level quality issues.

• 3+ years of experience with Playwright and automated testing frameworks.

• Strong expertise in Python (our primary automation language).

• Experience with CI/CD tools (Git Hub Actions or similar).

• Proficiency in version control systems (Git).

• Experience testing distributed systems.

• Understanding of modern software development practices.

• Experience working with cloud platforms (GCP preferred).

• Experience with runtime monitoring and testing of distributed systems.

• Knowledge of performance testing and APM (Application Performance Monitoring).

• Experience with end-to-end testing of complex, enterprise-grade applications.

• Understanding of distributed tracing and monitoring.

• Experience with chaos engineering and resilience testing.

This isn't a traditional QA role. You'll be at the forefront of a category-defining company, building testing infrastructure for AI-generated software that is transforming how Fortune 500 enterprises build products. You will have direct impact on the quality and reliability of an autonomous development platform moving at unprecedented speed—where your frameworks determine whether AI-built software is production-ready  challenges are novel, the stakes are high, and the team is world-class.

Led by two pioneering co-founders we are one of the fastest growing companies in the U.S., creating our own category of enterprise autonomous software development. We automate thousands of hours of software development for our customers, which includes strong representation within the…