Senior Inference SW Engineer
Listed on 2026-10-11
-
Software Development
AI Engineer (Applied/Software), Machine Learning/ ML Engineer
Design, implement, and operate major components of the LLM serving runtime and its Kubernetes orchestration layer, improving latency, throughput, GPU utilization, and distributed execution. Evaluate emerging inference technologies, resolve customer issues, and contribute through code reviews, mentoring, and strong engineering practices.
Requirements: Requires at least 8 years of software engineering experience, including 1–2 or more years working directly on LLM inference runtimes or production model serving; a degree in Computer Science or a related field is also specified. Candidates should have strong knowledge of inference engines and internals, Kubernetes, Go and Python, and be able to debug and profile C++/CUDA workloads.
Key
Skills:
LLM Inference, Inference Runtime Engineering, vLLM, SGLang, TensorRT-LLM, Continuous Batching, KV Cache Management, Kubernetes, Go, Python, C++, CUDA, Tensor Parallelism, Pipeline Parallelism, NCCL, GPU Profiling
Benefits: Health And Wellbeing Benefits, Personal And Professional Development
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).