Software Engineer, AI Runtime Systems

Staff Software Engineer, AI Runtime Systems About Crunchyroll

Founded by fans, Crunchyroll delivers the art and culture of anime to a passionate community. We super-serve over 100 million anime and manga fans across 200+ countries and territories, and help them connect with the stories and characters they crave. Whether that experience is online or in-person, streaming video, theatrical, games, merchandise, events and more, it’s powered by the anime content we all love.

Join our team, and help us shape the future of anime!

Crunchyroll's Platform Development organization powers the infrastructure that delivers anime at scale to millions of fans worldwide. We are seeking a Staff Software Engineer to join our team in Los Angeles.

In this role you will drive the design and evolution of core platform services that power Crunchyroll's global ecosystem. Your work will span authentication and security enhancements, notification services, and ML inference runtimes, forming the foundation that enables engineering teams to build reliable, secure, and intelligent experiences at scale.

You will lead architectural initiatives, define technical direction, and ensure system scalability, performance, and resilience across distributed environments. Partnering closely with ML, data science, and engineering teams, you will shape the platform capabilities that support deploying and operating models in production, ensuring they meet the reliability and efficiency standards required for a global streaming service.

In the role of Staff Software Engineer, you will report to the Engineering Manager, Platform.

• Architect, build, and maintain ML inference runtimes for multi-model serving, autoscaling, and GPU/TPU utilization.
• Optimize inference pipelines and platform services for performance, reliability, and scalability.
• Lead deployment, operationalization, and maintenance of ML workloads in collaboration with ML and data science teams.
• Shape and maintain core platform services, including authentication, security, and notifications.
• Ensure seamless integration with platform infrastructure, CI/CD pipelines, and observability systems.
• Define scalable system architectures and guide cross-team design alignment.
• Develop benchmarking, validation, and monitoring tools to measure and maintain system performance.
• Promote security, compliance, and engineering best practices across platform and ML services.
• Mentor and influence engineering peers, fostering technical excellence and consistent standards.

• 12+ years of backend software engineering experience, with a track record of leading complex projects end-to-end.
• Hands-on experience building and optimizing AI/ML inference runtimes (e.g., KServe, Torch Serve, Tensor
  
  RT, Triton) and integrating with CI/CD and MLOps pipelines (e.g., Sage Maker, Kubeflow, Bento
  
  ML).
• Experience with containers, orchestration (Kubernetes/ECS), cloud platforms (AWS preferred), and distributed systems.
• Experience with performance profiling, model optimization, GPU acceleration, and designing inference workloads to meet latency/throughput SLAs.
• Experienced in building scalable APIs (REST/gRPC), caching strategies, and high-performance systems, including relational and No
  
  SQL databases.
• Familiar with monitoring, observability tools, security, and compliance best practices in production ML/AI services.
• Proven ability to collaborate with ML/AI teams, bridge research and production, and mentor peers.
• Strong problem-solving, communication, and engineering best practices, with attention to detail and quality.
• Bachelor's degree in Computer Science, Engineering, or a related field—or equivalent practical experience.

We are a dedicated Platform Development team building foundational services that enable engineering teams to deliver features faster and more reliably. Our mission is to create scalable, reusable, and high-quality systems across Crunchyroll, including core services, authentication/security enhancements, notifications, and ML inference runtimes. We leverage cloud-based microservices architectures and best practices to deliver reliable, maintainable, and high-performance services for Crunchyroll's global audience.

In addition to collaborating with fun, passionate colleagues, you will enjoy the following benefits and perks:

• Competitive compensation package with salary plus performance bonuses.
• Flexible time off policies to support work-life balance.
• Comprehensive health, dental, and vision coverage, plus life insurance.
• Health Savings Account (HSA) and FSA programs.
• 401(k) plan with employer match.
• Parental leave and supportive programs for new parents.
• Pet insurance and pet-friendly offices where applicable.

Life at Crunchyroll: hybrid schedule in Los Angeles with in-office days Tuesday–Thursday.

Crunchyroll is an equal opportunity employer. We value diversity and do not discriminate on the basis of race, religion, color,…