R · via a16z Speedrun 1d ago Principal Engineer, Model Runware; Picfinder) US Express interest

Position: R Open role · via a16z Speedrun 1d ago Principal Engineer, Model Performance Runware (Picfinder) US Express interest →
Runware (Picfinder)# Principal Engineer, Model Performance Remote-friendly United StatesAT A GLANCE Lead inference performance optimization across a distributed GPU fleet serving 400k+ models, owning kernels, serving engines, and model-level optimizations.

LEVEL staff
· 10+ yrsKEY SKILLSCUDAGPU architecture

TensorRT-LLMvLLMSGLang Py Torch C++Python Triton Quantization Tensor  parallelism

Speculative decodingNCCLNsight SystemsWHAT YOU'LL DO
* Own end-to-end inference performance for the fleet across image, video, audio, 3D and LLM workloads
* Set the model performance roadmap and prioritize optimization efforts based on traffic and hardware
* Write and tune CUDA kernels, fuse attention/MLP paths, implement quantization formats (FP8, FP4, INT4)
* Build pipeline to optimize newly released models from weights to production deployment in days
* Choose and shape the serving stack (TensorRT-LLM, vLLM, SGLang, custom pipelines)
* Own benchmarking and performance regression system with CI integration
* Partner with infrastructure and hardware teams on GPU memory, storage and network bottlenecks
* Build and lead the Model Performance team, hire senior engineers, set technical standardsTHE ROLE Runware is the AI inference platform that builds and runs its own infrastructure, end to end. Our Sonic Inference Engine has served more than 20 billion requests for over 1 million developers and 500 million end users, with 400,000+ models behind a single integration, at up to 10x lower cost per token than traditional providers.

We are now building the AI Factory that runs it. The Sonic Inference Pod packs 1 MW of custom-designed compute into a 20-foot container with closed-loop, water-free cooling, sited wherever power exists. Pods are live in the US and Europe, and we are scaling to dozens of pods and tens of thousands of GPUs, all operated as one distributed fabric that reroutes traffic the moment a pod goes offline (announcement).

Every request that hits that fleet, whether it is an image, a video, a voice or a token stream, has to come back faster and cheaper than anyone else can serve it. Model performance is what makes that true. That is the job. The role As Principal Engineer, Model Performance, you will own how fast and how cheaply every model on Runware runs: diffusion and video generation, speech, 3D and LLMs, across 400,000+ models and tens of thousands of GPUs.

You will set the performance roadmap, build the hardest pieces with your own hands, and build and lead the Model Performance team as it grows around you. This is a principal-level role for someone who has led inference performance work at the frontier and still writes the kernel. You will decide where the next 2x comes from on a given model and hardware generation, whether that is a fused attention kernel, a better quantization recipe, speculative decoding, a smarter batching policy or a different parallelism layout, and then make it land in production on real traffic.

You will work directly with the Sonic Inference Engine team, the infrastructure and hardware teams, and with NVIDIA and the open-source engine communities, and you will hire, mentor and lead the engineers who do this work alongside you. Success means Runware serves a new open model at the Pareto frontier of latency and cost within days of its weights being released, our cost per request keeps falling with every hardware generation, and the team you have built can do this without you in the loop.

What you'll do
* Own inference performance end to end for the fleet: the kernels, the serving engines, the model-level optimizations and the benchmarks that prove them, across image, video, audio, 3D and LLM workloads on every GPU generation we run.
* Set the model performance roadmap.…