Site Reliability Engineer

AI Fabrik builds an edge inference delivery network for high-performance tokens, with faster time-to-market from grid to tokens. Our mission is to build the inference infrastructure we wished every enterprise already had — close to users, close to the cloud, and extremely resilient for real-time workloads. We are builders, architects, engineers, and researchers with hands-on experience in real-world AI deployment in production, and decades of data center experience that taught us exactly what needs to change.

AI Fabrik was incubated inside Gruve and backed by Mayfield, Xora (Temasek), Acclimate Ventures, Cisco Investments — existing investors from Gruve who followed us into this new chapter. We are deploying five initial production sites, with the first one coming online in July 2026.

We're looking for an experienced Site Reliability Engineer to build and operate the infrastructure powering next-generation AI inference platforms. In this role, you'll own the reliability, scalability, and performance of a globally distributed GPU infrastructure that serves production workloads with high availability and low latency.

• Infrastructure as Code (IaC): Proficiency in automating infrastructure provisioning with Terraform and cluster configuration management. Experience managing k8s infrastructure on-site and at hyperscalers is a must.
• Networking & Connectivity: Proficiency in L4/L7 load balancing, API gateways, and BGP/Anycast routing configurations to manage traffic and failover across distributed edge locations.
• Kubernetes Mastery: Deep expertise in managing production-grade Kubernetes s includes cluster lifecycle management, advanced scheduling (using taints, tole rations, node affinity, and custom schedulers for GPU workloads), and troubleshooting control plane/data plane failures.
• Nvidia GPU Stack: Hands on experience (preferred) or demonstration of understanding of Nvidia hardware (H100/B200/RTX
  6000) and the software stack required for high-performance inference. This includes drivers, CUDA, NVLink configurations, and working with libraries like TensorRT-LLM, vLLM, or SGLang.
• System Reliability at Scale: Experience implementing 99.99% uptime strategies, including regional failover, automated remediation playbooks, and self-healing systems.
• Observability Stack: Deep experience with metrics, logging, and distributed tracing (e.g., Prometheus, Grafana, Open Telemetry, Loki) and building dashboards for 24x7 visibility and alerting.
• Zero-Trust Security: Experience with hardening Kubernetes clusters and implementing network policies to ensure data isolation between multi-tenant customers and security patching with no customer impact.

• Inference Serving Optimization: Understanding of inference-specific performance levers, including continuous batching, KV cache management, and quantization (INT8, FP8).
• Model Routing & Scheduling: Familiarity with concepts like model sharding, request routing, and autoscaling.
• Agentic SRE: Experience in using/building agentic tools to perform SRE function e.g., automatically triage alerts with the needed context and apply/suggest remediations.
• Hardware Lifecycle Management: Ability to manage the full hardware lifecycle—from physical site bring-up and bare-metal OS provisioning (e.g., via Rafay) to firmware updates, driver patching, and RMA procedures—without disrupting live inference traffic.

At AI Fabrik, we hire for impact. We want those who challenge how inference infrastructure is built and who excel at delivering it in production. We are builders, architects, engineers, and researchers. We move fast, work with rigor, and care deeply about what runs in the real world.

We are committed to building a diverse and inclusive team. AI Fabrik is an equal opportunity employer. We welcome applicants from all backgrounds and thank all who apply; however, only those selected for an interview will be contacted.