Member of Technical Staff, Cluster Infrastructure

About us

We are building AI systems that can reason, use tools, and complete meaningful work in the real world. Our team works across model post-training, reinforcement-learning infrastructure, large-scale training, and product engineering. We believe the fastest path to more capable and reliable agents is an integrated loop: challenging environments, rigorous evaluations, efficient training, reliable inference, and products that make those capabilities useful.

You will own systems across the lifecycle of our accelerator clusters—from bringing capacity online and upgrading fleets to detecting failures, recovering safely, and retiring capacity. Your work will determine how quickly researchers can start experiments, how efficiently expensive hardware is used, and how reliably long-running workloads complete.

This is a hands-on infrastructure role spanning cloud and datacenter environments, cluster control planes, networking, storage, security, observability, and automation. You will partner with hardware and cloud providers and our Training, RL Systems, Post-Training, Inference, and Security teams to turn heterogeneous compute into a dependable platform. Depending on experience, you may lead multi-quarter initiatives and help set technical direction.

• Design, build, and operate control-plane services and infrastructure-as-code for provisioning, configuration, validation, upgrades, expansion, draining, recovery, and decommissioning; make every change repeatable, auditable, and safe to roll back.

• Bring new accelerator capacity online on schedule by coordinating dependencies across cloud providers, datacenter and hardware partners, networking, storage, security, and internal compute consumers.

• Build high-bandwidth, topology-aware connectivity within and across clusters; diagnose performance and reliability issues spanning hosts, switches, routing, transport, collective communication, and workload placement.

• Make clusters secure by default through identity and access controls, network policy, workload isolation, host and container hardening, secrets management, and trusted software and image supply chains.

• Improve fleet scalability, consistency, and fault tolerance by defining health signals, automating remediation, reducing configuration drift, and designing for partial failure.

• Establish service-level objectives and observability for cluster readiness, provisioning time, usable capacity, job-start latency, infrastructure-caused failures, utilization, and recovery time.

• Lead incident response and blameless postmortems for cluster failures; turn recurring operational pain into automation, safer defaults, and simpler system boundaries.

• Work directly with Training, RL Systems, Post-Training, and Inference engineers to debug cross-layer failures and shape a long-term compute, data, networking, and capacity roadmap.

• Deep expertise in distributed systems, reliability, and cloud platforms (e.g., Kubernetes, IaC, AWS/GCP/Azure).

• Strong programming ability in Python, Go, Rust, or another language suited to reliable infrastructure services and automation.

• Hands-on experience with Linux, containers, Kubernetes or another cluster scheduler, infrastructure-as-code, and at least one major cloud platform or substantial bare-metal environment.

• A practical understanding of networking, storage, identity, observability, and reliability, with the ability to trace a failure across multiple layers of a complex system.

• Experience designing systems for safe rollout, fault isolation, idempotency, capacity growth, and recovery from partial or large-scale failures.

• High ownership and clear communication, including comfort coordinating multi-team projects and participating in a healthy on-call rotation.

• Experience operating large GPU or accelerator fleets for distributed model training, inference, scientific computing, or another communication-intensive workload.

• Depth in Kubernetes internals, custom controllers or operators, device plugins, cluster autoscaling, scheduler extensions, Slurm, or comparable orchestration systems.

• Experience with high-performance…