AI Inference Platform Engineer

AI Infrastructure | Model Serving | GPU Computing | Inference Engineering | ML Platforms

Our client, a growing AI Infrastructure company based in Berlin, is looking for an AI Inference Platform Engineer to build and optimise the platform used to serve production AI models across GPU-enabled infrastructure.

You'll work at the intersection of AI Infrastructure, Distributed Systems, and Platform Engineering
, focusing on inference performance, GPU utilisation, autoscaling, latency, and reliability.

• Build and operate Kubernetes infrastructure for production AI inference
• Deploy and optimise model-serving workloads using vLLM and NVIDIA Triton
• Improve GPU utilisation, throughput, and inference latency
• Design autoscaling strategies for dynamic AI workloads
• Build platform tooling and automation in Python
• Provision and manage infrastructure using Terraform
• Develop observability across models, GPUs, Kubernetes, and serving infrastructure
• Profile and troubleshoot inference performance bottlenecks
• Improve batching, concurrency, caching, and resource allocation strategies
• Build reliable deployment workflows for new models and model versions
• Partner with ML Engineers to move models efficiently into production

• 4+ years in AI Infrastructure, ML Infrastructure, MLOps, Platform Engineering, or similar roles
• Strong understanding of Linux and distributed production systems

• CUDA
• NVIDIA GPU Operator
• Py Torch
• KServe
• Ray Serve
• LLM inference optimisation
• Multi-GPU inference
• AWS / GCP GPU infrastructure
• Experience operating high-throughput or latency-sensitive inference services