FF SRE, AI Inference Engineer Ffive · Locations · Site Reliability –ago

At F5, we strive to bring a better digital world to life. Our teams empower organizations across the globe to create, secure, and run applications that enhance how we experience our evolving digital world. We are passionate about cybersecurity, from protecting consumers from fraud to enabling companies to focus on innovation.

Everything we do centers around people. That means we obsess over how to make the lives of our customers, and their customers, better. And it means we prioritize a diverse F5 community where each individual can thrive.

The
AI Inference Engineer
plays a critical role in the AI lifecycle by bridging the gap between high-performance model development and optimized deployment environments. This position focuses on optimizing
Large Language Models (LLMs)for inference, serving diverse environments—from GPU-rich data centers to resource-constrained edge devices with a strong emphasis on maximizing throughput, minimizing latency, and maintainingmodel accuracy.

This role is pivotal in advancing F5’s AI capabilities, ensuring enterprise-grade reliability byleveraginghardware acceleration, designing scalable infrastructure, andmonitoringsystem performance.

• Build andmaintainrobust inference engines using tools like
  vLLM
  ,
  TGI (Text Generation Inference), and
  NVIDIA Triton
  , ensuring high performance at scale.

• Handle deployment optimizations to deliver low-latency AI serving solutions for multiple business applications.

• Profile andoptimizemodels for specialized hardware backends, including
  NVIDIA GPUs
  (CUDA/TensorRT),
  Apple Silicon (CoreML), and AI accelerators like
  TPUs
  and
  LPUs
  .

• Collaborate with hardware teams to maximizeutilizationand performance across various computational environments.

• Design and implement
  auto-scaling architectures
  for online (real-time) and batch inference pipelines,leveraging
  Kubernetes
  for inference routing and orchestration.

• Ensure software solutions areoptimizedfor peak performance during traffic spikes,maintaining reliability and scalability.

• Establish robust observability frameworks tomonitor
  Time to First Token (TTFT), tokens per second, and memory bandwidthutilizationagainst service-level agreements (SLAs).

• Build and execute
  performance and load testing suites
  toidentifybottlenecks and ensure consistent reliability at scale.

• Programming
  
  Languages:
  
  Proficiency in programming languages such as
  Python
  ,C++,
  Rust
  , or
  Golang
  specifically for high-performance AI workflows.

• Inference Tools:Proven hands-on experience with tools like
  vLLM
  ,
  TensorRT
  ,
  Llama.cpp
  , and
  Ollama
  for inference development and optimization.

• Infrastructure Expertise:Strong familiarity with infrastructure technologies, including
  Docker
  ,
  Kubernetes
  , and cloud platforms such as
  AWS
  ,
  GCP
  , and
  Azure
  .

• Hardware Optimization Expertise:Comprehensive understanding of GPU and AI hardware, including techniques for profiling andoptimizingperformance for accelerators like NVIDIA GPUs and TPUs.

• Prior experience deploying
  Large Language Models (LLMs)with advanced techniques like
  Speculative Decoding
  or
  Paged Attention
  .

• Contributions to
  open-source inference libraries
  or hardware-level kernel development (e.g., CUDA, Triton kernels).

• Background in
  MLOps
  or
  SRE
  roles focused on
  high-performance AI endpoints
  and reliability during demand surges.

• Proficiency in designing scalable solutions for high-throughput inference environmentsoptimizedfor traffic bursts.

• Latency Reduction:Continuously improve inference latency metrics, ensuring minimal
  Time to First Token (TTFT)andmaximumtokens per second.

• Cost Efficiency:Ac…