Device Lab and Benchmark Engineer

Every model we ship needs latency, memory, and accuracy numbers from real devices. Build the lab and the tools that produce those numbers.

• Build the device lab and the benchmark system behind every number we publish.
• You’ve built tools engineers use every day, and know what skews a mobile benchmark.
• You live in or near Amsterdam, where the lab is.

Location Amsterdam

Type Full time

Customers compare our models by the benchmarks we publish. A phone runs a model more slowly when the phone is warm or charging, so the lab controls for temperature, charging, and background load before a number goes public.

You also report where a model runs slowly, with a profile and a proposed fix for the model and runtime engineers. You test our models on hardware we don't support yet, including new phones and robots. You build demo apps with go-to-market and marketing that show a model running on a specific device.

• Keep an inventory of all team hardware: laptops, phones, tablets, and training machines.
• Send bottleneck reports to the model and runtime engineers, with a profile and a proposed fix.
• Build demo apps and examples with go-to-market and marketing.
• Turn results into charts, model card sections, and blog posts.
• Run our models on new hardware, from the latest phones to robots.

• Know what skews a mobile benchmark: thermal throttling, background load, warm versus cold runs, and charging state.
• Have measured models on Core ML, LiteRT, or ONNX Runtime and can explain why results vary between runs.
• Live in or near Amsterdam, where the lab is.
• Write clean, tested code, and spot a weak change in review, whether a person or an agent wrote it.
• Plan and run your work through coding agents such as Claude Code, with a low tolerance for slop.

Start what needs starting without waiting to be asked, and finish what you start. Take on work outside your role when a project needs you.

We ship quickly, so we cut scope until only the part users notice is left. Anyone can comment on your work or redo your draft, and we say early when work isn't ready. We read that feedback as help.

We judge the work by what shipped and what changed because of it. Nobody counts hours.