Sr. Backend Software Engineer; Infrastructure ML Ops

Position: Sr. Backend Software Engineer (Infrastructure ML Ops)
Job-f90da-95e4-4d7f-9917-3b192e12baea

Location:

San Francisco, California, United States;
Brooklyn, New York, United States;
Cambridge, Massachusetts, United States

Employment type:

FULL TIME Posted:
At Philo, we’re a group of technology and product people who set out to build the future of television, marrying the best in modern technology with the most compelling medium ever invented — in short, we’re building the TV experience that we’ve always wanted for ourselves. In practice this means leveraging cloud delivery, modern tech stacks, machine learning, and hand-crafted native app experiences on all of our platforms.

We aim to deliver a rock solid experience on the streaming basics, while cooking up next generation multi-screen and multi-user playback experiences.

Senior Backend Software Engineer (Infrastructure MLOps)
The infrastructure team at Philo builds the foundation for a 24/7 live and on-demand streaming product serving video to our growing subscriber base. It provides a platform that enables groups across the company – such as video delivery, product, support, and analytics – to do their best work easily and smoothly. Our systems are highly distributed, horizontally scalable and run in the cloud on Kubernetes.

We continuously test and deploy our code and closely monitor the health and performance of our production environment.

We are seeking an enthusiastic Infrastructure MLOps Software Engineer to help us build and improve the ML infrastructure  are building out ML and LLM infrastructure and this role will be pivotal in setting direction, adopting best practices, and shipping infrastructure that will be used across the company.

As a sample, the ML infrastructure supports teams like Discovery in building content recommendation models that will help our users find more favorite shows from the large Philo catalog. The engineering team is designing and developing our own agent harnesses to support coding and empowering everyone at the company to get answers across our code bases, analytics, logs, and communication tools.

The video player team is using LLMs to classify ads and label our video catalog so that users have a better experience. You’ll build the serving, evaluation and platform infrastructure that these workloads depend on.

Responsibilities:

Manage the core infrastructure at Philo with a focus on supporting ML and LLM workloads. This infrastructure is relied on by a large subscriber base of Philo’s product as well as internal stakeholders across the company.

Design, build, and maintain cloud infrastructure components (AWS and Kubernetes) using a combination of in-house technology and open source software.

Build and scale model serving and inference infrastructure for both real-time recommendation models and LLM workloads — including model servers (e.g. NVIDIA Triton), LLM inference engines (e.g. vLLM, TGI, TensorRT-LLM), autoscaling, and latency/throughput optimization. Manage GPU and accelerator infrastructure on Kubernetes — scheduling, capacity and quota management, and cost optimization across on-demand and spot capacity.

Design, build, and maintain monitoring and observability systems — spanning both system health and model quality (drift, evaluation, and online performance) — that enable Philo engineers and the support team to gain insight and to discover and debug issues.

Build and operate model evaluation infrastructure: offline and online eval pipelines, golden/regression datasets, LLM-as-judge harnesses, A/B testing and experimentation, and feedback/ground-truth loops

Design, build, and maintain CI/CD systems enabling Philo engineers to create pipelines to test and deploy their code.

Establish tools, methods and best practices for other Philo engineers interfacing with…