Senior Staff Engineer - AI ADC

Senior Staff Engineer - AI ADC

A global technology client of Insight Global is seeking a Senior Staff Engineer - AI ADC to join their team. This company specializes in high-performance application delivery, network security, and AI-ready infrastructure solutions for enterprises, service providers, and government organizations.

This role combines deep domain expertise in Application Delivery Controllers (ADC), Server Load Balancing (SLB), DNS/GSLB, with practical experience in AI/LLM systems, to build intelligent routing, inference-aware load balancing, and Layer-8 contextual decision engines for AI workloads. The ideal candidate is a senior staff engineer who can drive architecture, design, prototyping, and execution—working closely with product management, platform teams, and AI strategy groups.

Key Responsibilities Include:

-Lead architecture and development of AI Gateway components for intelligent routing of LLM/AI application traffic.

-Design and implement LLM-aware load balancing, incorporating semantic, token, latency, and model-level insights into traffic decisions.

-Develop enhancements for ADC and GSLB platforms to support AI workloads, including:

Token-aware rate limiting

Inference latency-based routing

AI model endpoint discovery and health checks

Integration with vector databases, model registries, and ML observability systems

-Contribute to advanced features including Layer-8 context-aware routing, adaptive traffic shaping, and AI-driven anomaly detection.

-Drive system-wide architecture: data plane, control plane, configuration, and distributed state management.

This role is permanent and will be worked hybrid on-site in San Jose, CA. We are looking to pay between $180,000-$195,000 base salary.