Software Engineer - LLM Inference

The Opportunity

When people talk about generative AI and other ML-powered solutions in today's conversation, they often refer to generative pre-trained transformers like ChatGPT that can respond to queries from a position of deep learning. A GPT-in-a-box solution removes the burden of building or implementing these AI solutions yourself. It also makes overcoming the complexity, inefficiency, and security challenges of generative AI and AI/ML applications easy.

Nutanix simplifies your learning curve on AI-ready infrastructure with Nutanix Cloud Platform for AI (GPT-in-a-Box). This high-performant Machine Learning full-stack cloud platform helps you optimise IT costs with a software-defined cloud operating model. Harness AI-ready capabilities right out of the box, simplified to build, fine-tune, and run models, including GPTs and LLMs, while you continue to use existing teams and skills.
Join the Nutanix AI team, responsible for the magic behind the scenes.
About the Team:
The Nutanix Enterprise AI team is responsible for strategic product areas including LLM Inference and the AI Gateway. We are at the forefront of Nutanix's mission to simplify AI deployment, recently showcasing our Agentic AI platform at NVIDIA GTC and NEXT 2026. This team is fast-paced, globally distributed, and focused on building the foundational layers of the AI stack.

You will report to a seasoned Technical Manager who will provide mentorship and guidance as you navigate through your responsibilities. The work setup at Nutanix AI is a hybrid model, offering a blend of in-office collaboration and remote work flexibility. As a new hire, you will be expected to be in the office for 3 days a week, ensuring that you have the opportunity to engage with your team and foster strong working relationships.

Your Role:

Architect, design, and develop horizontally scalable, containerized, fault-tolerant services on Kubernetes.

Improve the performance of systems to deliver for low-latency and high-throughput use cases.

Optimize any part of the stack, including low-level systems.

Leverage and contribute to relevant open-source cloud native projects.

Develop scalable, efficient, and fault-tolerant observability architectures for collecting, analyzing, and reporting metrics for various platform services.

Collaborate closely with globally located product management and backend development teams to deliver high-quality products in a fast-paced environment.

Contribute to all stages of the product development cycle: technical design, development, test, experimentation, analysis, and launch.

Be a team player by reviewing code and design docs, giving feedback on product specs and mocks, and documentation.

Participate in an ongoing process definition and technology selection to ensure our technology stack is current with relevant trends.

Continuously learn and improve your technical and non-technical abilities.

What You Will Bring

2-5 years of experience developing maintainable, modular, resilient, fail-safe, and long-lasting code from a Product Development company.

Have strong programming fundamentals, data structure, and algorithms.

Strong experience in Docker, Kubernetes, and Cloud native technologies

Experience building applications with Go and Python

Experience building and managing CI/CD pipelines

Strong understanding of datacenter design, including computing, storage, and networking.

Familiarity with on-prem, cloud, and hybrid software deployment architectures

Good experience in designing and tuning high-performance system software

Strong understanding of distributed computing and storage architectures

Strong knowledge of OS internals, virtualization, application performance monitoring, compute storage, and networking management

Familiarity with machine learning concepts and popular frameworks (like Tensor Flow, PyTorch, etc) is a strong plus

Experience with hardware accelerators, such as GPUs, is a strong plus.

Experience working with large codebases or contributing to open source is a strong plus.

Experience in building multi-tenant services on a virtualized infrastructure is a solid plus.

Detail-oriented with a strong focus on quality, design, and user experience.

Inquisitive and highly motivated self-starter and problem solver with a drive to integrate, communicate, and work well with large projects and teams.

Track record of being reliable, responsible, and thorough.

Bachelor's/Master's in Computer Science or equivalent work experience

Learn More About the Technology:
Highlighted Benefits (Vancouver, Canada)
Retirement:RRSP with dollar-for-dollar matching up to 7% of base salary
Mental Health:Dedicated mental health coverage plus top-tier paramedical benefits
Family:Fully paid maternity and parental leave and generous bereavement leave, including time for the loss of a pet
Equity:RSUs and Employee Stock Purchase Plan at a 15% discount
Time Off:Company holidays, sick days, company wellness days, and vacation starting at 10 days
Work Arrangement Hybrid:This role operates in a hybrid…