AI Inference Framework Engineer

Job-f5cd70-58f7-4c09-a8a0-e9254ceecddf

Location:

Burlingame, CA, United States

Employment type:

FULL TIME Posted:
Quadric is redefining edge AI with the industry's first General Purpose Neural Processing Unit (GPNPU), enabling developers to run both neural network inference and conventional C++ code on a single programmable architecture. Our technology powers intelligent edge devices across automotive, industrial, robotics, and embedded systems.

Founded in 2016 and based in downtown Burlingame, California, Quadric is building the world's first supercomputer designed for the real-time needs of edge devices. Quadric aims to empower developers in every industry with superpowers to create tomorrow's technology, today. The company was co-founded by technologists from MIT and Carnegie Mellon, who were previously the technical co-founders of the Bitcoin computing company 21.The

Opportunity Quadric's GPNPU runs both neural networks and C++ code on a single programmable architecture. As a member of our AI Application team, you will make the GPNPU a first-class backend in the frameworks developers already use, such as llama.cpp, PyTorch, and vLLM. That way, customers can move their applications from GPUs to Quadric with minimal changes. You'll own backend integration from memory and dispatch through serving features, working closely with our kernel, runtime and compiler teams.

What You'll DoIntegrate GPNPU backend into AI compute frameworks (llama.cpp, PyTorch, vLLM, Optimum)
Implement framework backend interface, memory allocation, compute dispatch, registration and operators/kernels

Implement model compute pipelines including prefill, decode, batching, multi-sequence and multi-modal

Support important framework serving features including KV-cache management, paged attention, continuous batching, speculative decoding

Optimize framework performance and accuracy for GPNPU backend

Reduce host-device dispatch overhead, implement graph capture/replay, integrate efficient kernels, minimize latency

Support quantization formats such as GGUF (Q4/Q8), GPTQ/AWQ, int8/fp16

Validate accuracy against reference backends (perplexity, task benchmarks) and drive improvements

What Success Looks Like Within your first 6-12 months, you’ll:

Bring up a new LLM/VLM family end to end on the GPNPU backend, with accuracy matching the reference

Deliver measurable tokens/s or latency gains on a frontier model

Ship one serving feature (e.g. paged KV cache) in a production release

What We're Looking ForRequired3+ years building or extending ML frameworks or inference runtimesBS/MS in CS/CE or related fields

Strong C/C++ and Python Solid understanding of framework internals: tensor memory management, graph scheduling, operator dispatch

Solid understanding of AI compute performance analysis and optimization

Fluent with AI coding agents (e.g. Claude Code, Cursor), and interested in building agent-driven workflows for backend and model bring-upPreferred

Experience writing an accelerator backend (CUDA, Metal, Vulkan, Hexagon or another NPU) for an open-source framework

Contribution to open-source frameworks (PyTorch, llama.cpp, vLLM)
Understanding of low level AI compute pipelines, CPU/NPU, DMAUnderstanding of graph compilers and kernel fusion

What We Offer The base salary range for this position is $160,000 to $195,000. This range reflects the full span of levels and geographies at which Quadric hires for this role. The actual base salary offered will depend on a number of factors, including the specific level of the role, years and depth of relevant experience, technical skills and competencies, the criticality of the role to the business, internal equity, and work location.

In addition to base salary, this role is eligible for equity and a…