Senior/ML Engineer
-
Software Development
Job Description & How to Apply Below
Cavendish Professionals are looking to speak to Senior ML Inference Engineers to join a team working on real-time voice AI at genuine consumer scale, supporting hundreds of millions of users and processing over 10 trillion tokens per month.
This is an opportunity to take full ownership of inference optimisation, from research through to production, within a company of approximately 50 people with top-tier backing.
The role offers strong compensation with equity, and is based in Mountain View, CA, with remote working possible from the UK, Germany, Switzerland and Serbia.
The successful candidate will have extensive experience in machine learning inference optimisation and systems engineering, with strong expertise in GPU performance tuning, distributed systems and cloud-native technologies. They will be responsible for taking models from the research team, containerising them, optimising serving for sub-second, high-concurrency workloads and ensuring reliability in production environments running on NVIDIA GPUs.
Skills and Requirements:- Senior level or above (equivalent to Google L5 / Meta E5)
- Hands-on experience with ML inference optimisation using vLLM or TRT-LLM, with a minimum of 5 years' in relevant tech. The client are also looking for experience building backend/ML systems and serving models in production.
- Strong systems programming and GPU optimisation experience using C++, CUDA, Rust or highly optimised Python.
- Experience with model acceleration techniques, including:
Quantisation, Distillation, Speculative decoding, KV/caching strategies, Continuous batching, Paged attention. - Experience with distributed systems and large-scale infrastructure, including:
Kubernetes and Ray, Custom load balancing, Multi-GPU and multi-node inference, Supporting thousands of concurrent connections. - Tech Stack: vLLM, TRT-LLM, CUDA, C++, Rust, Python, Kubernetes, Ray, Docker and NVIDIA GPUs
- Public contributions, including open-source contributions to inference engines or systems projects, and technical write-ups.
- PhD in Computer Science, Physics or Mathematics, or equivalent practical depth.
- Progression within an organisation is highly desired, rather than job hopping
- Ability to commute to Mountain View (For US candidates).
- Own inference optimisation end-to-end, taking models from the research team through to production.
- Containerise models and optimise their serving capabilities.
- Optimise model serving for sub-second, high-concurrency workloads.
- Ensure models remain reliable in production environments running on NVIDIA GPUs.
- Apply model acceleration techniques to improve inference performance.
- Profile and tune systems to address latency and throughput challenges.
- Work with distributed systems and infrastructure to support multi-GPU and multi-node inference.
- Build and maintain backend and machine learning systems that serve models in production.
Cavendish (Recruitment) Professionals Ltd are proud to be an equal opportunity employer and we believe that inclusivity begins with the candidate experience. All qualified applicants will receive consideration for employment regardless of, gender, race, age, sexual orientation, religion, or belief.
Position Requirements
10+ Years work experience(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction