Skip to content
×

Inference Acceleration Engineer Runtime Remote; US, EU

Remote / Online - Candidates ideally in
San Francisco, San Francisco County, California, 94199, USA
Listing for: Iframe
Remote/Work from Home position Listed on 2026-11-21
Job specializations:
Salary/Wage Range or Industry Benchmark: 220000 - 360000 USD Yearly USD 220000.00 360000.00 YEAR

Job Description & How to Apply Below

Position: Inference Acceleration Engineer Runtime Remote (US, EU)

Own end-to-end performance for one or more model families on our managed inference runtime. We are 10–20× faster than hyperscaler inference today; the bar is to widen that gap on every release.

Hiring manager replies within 5 business days.

About the team

The runtime team is the engineering side of the research lab. Six engineers today. We write the kernels, the scheduler, and the serving layer for managed inference. The runtime ships into customer production weekly.

Reports to the head of runtime. One on-call rotation shared across runtime, cluster SRE, and customer engineering.

What you'll do
  • 01

    Profile real customer workloads with Nsight, rocprof, and our internal tracer; turn the bottleneck into a paper-quality benchmark and a merged kernel.

  • 02

    Write fused, attention-aware kernels in CUDA and Triton (and ROCm/HIP for AMD MI300X) targeting H100, H200, B200, and B300.

  • 03

    Own one model family end-to-end: tokenizer through KV cache through speculative decoding, on every supported accelerator.

  • 04

    Design and ship the next round of long-context primitives — paged KV, ring attention, sliding-window cache eviction.

  • 05

    Co-author at least one external write-up per quarter: a blog post, a paper, or a kernel released to the open ecosystem.

  • 06

    Carry the runtime on-call rotation alongside cluster SRE and customer engineering — about one week per six.

What we're looking for

Five-plus years of systems-level performance work, with at least two of those years deep in GPU kernels.

Strong CUDA and at least one of Triton, CUTLASS, or HIP. Comfortable reading PTX and SASS when the profile demands it.

Track record of measurable speed-ups landed in production or in widely-used open source — bring a PR or a perf graph.

Comfort with transformer internals: attention variants, KV cache shapes, quantization (FP8, INT4 / NF4), speculative decoding.

Empirical instincts. You design experiments, hold variables fixed, and write down the numbers before the next change.

Nice to have, not required

Published kernels in vLLM, SGLang, TensorRT-LLM, Flash Attention, or similar.

Experience tuning collectives (NCCL / RCCL) and topology-aware scheduling.

AMD MI300X / MI325X experience — most candidates do not have this; we will weight it heavily.

Prior research experience or co-authorship on systems / ML papers.

Compensation In writing, like everything else

We publish bands. We meet them. The number you see on the offer is the same number your future peers got at the same level. We do not negotiate; we level.

Base

$220,000 – $360,000 USD (US), depending on level (IC4 / IC5 / IC6).

Equity

Meaningful early-stage equity, refreshed on tenure milestones, not review cycles.

Notes

Pay bands are published internally and shared in the manager call. We do not negotiate; we level.

To View & Apply for jobs on this site that accept applications from your location or country, tap the button below to make a Search.
(If this job is in fact in your jurisdiction, then you may be using a Proxy or VPN to access this site, and to progress further, you should change your connectivity to another mobile device or PC).
View / Apply for Jobs
Matching My Jurisdiction

Search for further Jobs Here:

SEARCH JOBS: 10+ Million Listings
(Try combinations for better Results! Or enter less keywords for broader Results)
Education Level
Experience Level (years)
Posted in last:
Salary