AI Accelerator Software Principal Engineer – Runtime Library

Ampere is a semiconductor design company for a new era, leading the future of computing with an innovative approach to CPU design focused on high-performance, energy efficient AI compute.

As a pioneer in the new frontier of energy efficient high-performance computing, Ampere is part of the Softbank Group of companies driving sustainable computing for AI, Cloud, and edge applications.

Join us at Ampere and work alongside a passionate and growing team-we’dlove to have you apply!

As an AI Accelerator Principal Software Engineer – Runtime Library, you will lead the design, development, and optimization of AI runtime software that enables multiple state-of-the-art deep learning models to run efficiently on Ampere’s deep learning accelerators. You will work at the intersection of systems software, performance engineering, and AI enablement, helping deliver high-throughput, low-latency inference and a strong foundation for future model and framework support.

• Build and evolve an AI Runtime Library for Ampere accelerators that supports execution, scheduling, and lifecycle management of deep learning workloads across multiple model types and popular frameworks.
• Own end-to-end acceleration paths
  , going deep into the full SW/HW stack—including:
  • Inference serving and integration layers
  • Compiler/runtime interfaces and graph/IR execution flows
  • Runtime library architecture (APIs, memory management, operators, execution engines)
  • Communication mechanisms and device/host orchestration
• Drive HW/SW co-design and optimization to improve:
  • Throughput (tokens/requests per second)
  • Latency (kernel execution and scheduling efficiency)
  • Memory efficiency (buffering, paging, reuse, caching)
  • Overall compute utilization and scaling behavior
• Contribute to AI co-processor/accelerator software enablement
  , partnering closely with hardware and systems teams to ensure runtime and kernel strategies match accelerator capabilities and constraints.
• Collaborate cross-functionally to integrate runtime components into Ampere platform stacks, ensuring robust deployment on target environments and consistent performance in production-like workloads.

• BS Computer Science, Computer Engineering, Electrical Engineering, or Software Engineering or related technical field & 8 years of related experience; or MS degree & 6 years; or PhD & 3 years
• Proven experience developing user-mode drivers and/or runtime libraries for GPUs or deep learning accelerators in Linux or RTOS environments.
• Strong expertise in C/C++ and systems-level programming (memory, threading, synchronization, performance profiling).
• Demonstrated background in AI framework enablement, with hands‑on experience in one or more of:
  • PyTorch (operator/runtime integration, graph execution, correctness/performance work)
  • llama.cpp (inference/runtime execution patterns)
  • ONNX (graph handling, interoperability, execution engines)
• Strong performance engineering skills, including profiling/diagnostics and optimization of execution pipelines, data movement, and compute kernels.
• Ability to operate effectively in a collaborative environment—owning complex components while partnering with compilers, hardware, and platform teams.

At Ampere we believe in taking care of our employees and providing a competitive total rewards package that includes base pay, cash long-term incentive, and comprehensive benefits. The full base pay range for this role is between $182,000 and $273,000, except in the San Francisco Bay Area where the range is between $195,000 and $292,000.

Our benefits include health, wellness, and financial programs that support employees through every stage of life.

• P…