Principal AI Ops Engineer

Crux AI is a newly formed, U.S.

-based integrated AI infrastructure company created to remove the physical and operational constraints on consequential AI ambitions. Crux brings together power, high-density data centers, TPU silicon, networking, orchestration software, and ongoing operations as one integrated system.

Crux is being capitalized to plan every layer together, develop each one to demanding standards, and operate the whole system with efficiency and reliability. That gives hyperscalers, frontier AI labs, sovereign customers, enterprises, and AI-native companies greater freedom to pursue the AI they are here to create.

You will build the systems that let AI agents, not people, do most of the work of running Crux's TPU fleet. Reporting to our Director, AI Ops, you will be one of the first engineers on the team, writing the telemetry pipelines, correlation logic, and agents that detect, triage, and remediate failures across facilities, TPU hosts, the network, and the control plane before a customer notices.

You will turn every runbook into code and every repeat incident into automation. This is a hands-on engineering seat: you write and ship the software, then operate it in production.

• Build and own the telemetry and event pipeline that ties together power, cooling, TPU hosts, interconnect, and the control plane, with the correlation to tell a failing pump from a failing chip from a failing job.

• Write AI agents that triage alerts, execute runbooks, drain and requeue workloads, and open hardware repair and RMA requests with Google, with every action logged, reversible, and measured.

• Turn runbooks into code, so that every critical failure mode for the first build maps to a signal, an alert, an automated response, and an owner.

• Build the guardrails that keep autonomous actions safe: blast-radius limits, approval gates for high-risk changes, and tested rollback paths.

• Define and instrument SLOs and error budgets with Compute, Network Engineering, and Facility Operations, and build the views the team reviews every week.

• Take part in on-call and incident response, and drive postmortems that end in a fix that ships, not an action item that ages.

• Measure the share of operational work that closes with zero human touch, and drive that number up release over release.

• Gained a complete understanding of the first-build architecture, the failure modes that matter most, and the tooling in place today, and shipped a first working agent against a real, repeat failure.

• Stood up the first version of the unified telemetry pipeline for the first build.

• Adapted to a fast-paced environment, triaging appropriately and continuously improving our function.

• Taken the first Stage 1 cluster into production with automated detection and remediation in place for its top failure modes.

• Shipped agents the team relies on every day, with a measured share of incidents resolved without human intervention.

• Built the guardrails and audit trail that let the team trust autonomous actions in production.

• Running automation across a multi-site fleet where most repeat operational work executes autonomously, with measurable gains in availability, recovery time, and cost per chip.

• Built a library of agents and runbooks-as-code that every new site inherits on day one.

• Become a reason strong SRE and operations engineers choose Crux.

• 10+ years in software engineering, SRE, or production engineering for large-scale infrastructure, including operating a fleet across many sites against hard SLAs.

• Built…