Senior Inference Engineer

Aleroy is building a unified platform for high-performance AI inference, making it easier for companies to deploy and run AI models at scale.
Through a single API, Aleroy provides access to ready-to-use and custom AI models across cloud infrastructure, dedicated capacity, and customers' own hardware. Our platform optimizes performance, cost, and deployment flexibility while giving enterprises control over where and how their AI workloads run.
We work in small teams, collaborate directly, and focus on solving complex engineering challenges to build reliable, scalable AI infrastructure.

You will work across inference serving, GPU performance, distributed systems, benchmarking, and model optimisation to make our serving infrastructure faster and more efficient while maintaining a clear quality bar.
This is the senior technical seat for AI serving  will set the serving and measurement roadmap, establish engineering standards, and guide engineers working across serving, metrics, and testing.
The role is highly hands-on. You’ll be expected to work directly with the inference stack, identify performance bottlenecks, implement improvements, and demonstrate their impact through rigorous and reproducible measurement.

• Inference Serving: Own and improve the FAR Labs inference serving stack, including prefill/decode disaggregation, continuous batching, KV-cache management, cache-aware routing, speculative decoding, and other serving optimisations.

• Serving Performance: Improve throughput, latency, memory efficiency, and serving cost while maintaining clearly defined model-quality standards.

• GPU Performance Engineering: Identify and eliminate GPU performance bottlenecks across the inference stack using CUDA, Triton, profiling, kernel optimisation, memory optimisation, and appropriate precision strategies.

• Mixture-of-Experts Serving: Drive efficient serving of large MoE models, including expert parallelism, all-to-all communication, expert-load balancing, and distributed execution.

• Quantization & Model Optimisation: Implement and evaluate quantization and other model optimisation techniques while measuring accuracy against appropriate full-precision references.

• Benchmarking & Measurement: Own rigorous, reproducible benchmarking across serving performance, quality, and cost. Establish measurement methodologies that can withstand external technical scrutiny.

• Performance Instrumentation: Build and improve the instrumentation required to understand system behaviour, identify bottlenecks, compare configurations, and validate improvements.

• Distributed Serving: Make technical decisions around routing, scheduling, caching, and workload distribution across diverse hardware environments.

• Strong hands-on experience building, optimising, or operating LLM inference-serving systems in production.

• Deep understanding of inference stacks such as vLLM, SGLang, TensorRT-LLM
  , or similar technologies.

• Strong understanding of paged attention, continuous batching, disaggregated serving, KV-cache management, and modern inference architectures.

• Strong GPU performance engineering experience using CUDA or Triton
  .

• Strong understanding of GPU performance characteristics including memory bandwidth, compute utilisation, model bandwidth utilisation, FLOPs utilisation, and modern GPU precis ions such as FP8 and FP4.

• Experience serving large Mixture-of-Experts (MoE) models, including expert parallelism, all-to-all communication, and expert-load balancing.

• Hands-on experience with model quantization and measuring quality against full-precision references.

• Strong benchmarking experience, including load generation, latency percentiles, throughput-at-SLO, quality measurement, and reproducibility.

• Strong…