AI Infra Engineer - Model Inference Systems; Multimodal​/LLM​/VLM

AI Infra Engineer - Large Model Inference Systems (Multimodal/LLM/VLM)

Location:

San Jose

Employment Type:

Regular

Job Code: A122047

• Build and evolve next-generation inference systems for large-scale online traffic, including global scheduling across heterogeneous compute resources, high-concurrency load balancing, and efficient batch formation.
• Optimize distributed inference for 200B+ models and complex multimodal models through tensor parallel, pipeline parallel, data parallel, and related strategies to improve throughput and latency in production.
• Develop high-performance kernels for frontier model architectures such as MoE, emerging attention mechanisms, and multimodal fusion layers using CUDA, Triton, and related tools.
• Explore AI-driven infrastructure for inference systems, including AI agents for kernel optimization, performance tuning, consistency validation, deployment pipelines, and intelligent operations.

• Bachelor's degree or above in Computer Science, Software Engineering, Artificial Intelligence, Mathematics, or related fields.
• 2+ years of experience in high-performance computing, distributed scheduling systems, or large-model inference engine development.
• Familiarity with large-model architectures and strong system design skills for complex, high-concurrency environments.
• Strong understanding of asynchronous scheduling, resource pooling, and load balancing in distributed microservice systems.
• Strong engineering skills in performance optimization and production system development.

• Deep understanding of inference frameworks such as vLLM and SGLang, with hands‑on experience in customization and production optimization.
• Familiarity with GPU microarchitecture and operator‑level optimization using CUDA, Triton, Cutlass, or related tools.
• Experience with LLM inference optimization, such as PTQ, QAT, KV cache optimization, or PD disaggregation.
• Experience deploying and optimizing VLMs or multimodal models in production.

The base salary range for this position in the selected city is $156,000 - $387,600 annually. Compensation may vary outside this range depending on qualifications, skills, competencies, and experience.

Employees receive medical, dental, and vision insurance, a 401(k) plan with company match, paid parental leave, short‑term and long‑term disability coverage, life insurance, wellbeing benefits, 10 paid holidays per year, 10 paid sick days per year, and 17 days of paid personal time (prorated upon hire and accruing with tenure). The company reserves the right to modify these benefits at any time.

Qualified applicants with arrest or conviction records will be considered in accordance with all federal, state, and local laws, including the Los Angeles County Fair Chance Ordinance for Employers and the California Fair Chance Act. Criminal history may affect the following duties:

• Interacting and occasionally having unsupervised contact with internal/external clients and/or colleagues;
• Appropriately handling and managing confidential information including proprietary and trade secret information and access to information technology systems;
• Exercising sound judgment.

Employment is governed by Ecuadorian laws and the above EEO statement applies to all candidates in eligible jurisdictions.