Principal AI​/ML Platform Engineer - Remote

Improve the lives of others while Caring. Connecting. Growing together.

Job Description
- Principal AI/ML Platform Engineer
- Remote (2386255)

Optum Tech is a global leader in health care innovation. Our teams develop cutting-edge solutions that help people live healthier lives and help make the health system work better for everyone. From advanced data analytics and AI to cybersecurity, we use innovative approaches to solve some of health care’s most complex challenges. Your contributions here have the potential to change lives.

Ready to build the next breakthrough? Join us to start Caring. Connecting. Growing together.

• Own the end-to-end reference architecture for multi-tenant AI compute platforms across hybrid on-premises bare-metal Open Shift AI clusters and public-cloud managed AI platforms (Azure AI Foundry, AWS Bedrock, GCP Vertex AI)
• Set network, latency, and topology standards for distributed training (NVLink, Infini Band, RoCEv2, GPUDirect RDMA, NCCL), ensuring interconnect boundaries are strictly maintained
• Establish cluster governance, Git Ops workflows (Argo CD), RHACM policies, and automated lifecycle management for bare-metal accelerated compute nodes
• Design cost and utilization models including capex amortization, accelerator-sharing strategies (MIG/time-slicing), and cost-per-token/training-run math to inform accelerator procurement roadmaps
• Standardize model-serving platforms (vLLM, KServe) and inference gateways, setting quantization policies, provenance review gates, and intelligent model routing rules
• Define workload placement frameworks to determine self-hosted versus managed cloud deployment based on data residency, latency, cost, and compliance requirements
• Design and enforce identity, access, and security controls for AI workloads and autonomous agents, including least-privilege RBAC, Vault secret management, short-lived credentials, and mTLS
• Establish enterprise Service Level Objectives (SLOs), disaster recovery plans, and upgrade strategies for Open Shift, Open Shift AI, GPU operators, drivers, and firmware
• Partner with cross-functional AI teams, LLM gateway engineers, privacy, and security stakeholders to ensure seamless integration and HIPAA compliance

You’ll be rewarded and recognized for your performance in an environment that will challenge you and give you clear direction on what it takes to succeed in your role as well as provide development for other roles you may be interested in.

• Bachelor’s degree or 4+ years of equivalent software/platform engineering experience in lieu of a degree
• 10+ years of experience in infrastructure, Dev Ops, SRE, or ML platform engineering
• 5+ years of experience operating Kubernetes or Open Shift at scale in production bare-metal or enterprise cloud environments
• 3+ years of experience designing and managing accelerated-compute (AI/GPU) infrastructure utilizing NVIDIA GPU Operator, NFD, MIG/time-slicing, and DCGM
• 3+ years of experience architecting hybrid AI platforms spanning self-hosted IaaS and public-cloud PaaS managed AI services (e.g., Azure AI Foundry, AWS Bedrock, or GCP Vertex AI)
• 3+ years of experience with HPC/AI networking technologies, including Infini Band or RoCEv2, GPUDirect RDMA, and NCCL collective communication limits
• 3+ years of experience managing multi-cluster fleets using RHACM (or equivalent) and Git Ops tooling (Argo CD or Flux)
• 3+ years of experience implementing enterprise security and IAM controls for software workloads (RBAC, OIDC/OAuth, Vault secrets management, mTLS)

• Experience with distributed training frameworks (PyTorch DDP/FSDP, Deep Speed, Ray, JAX) and batch scheduling systems (Kueue, Volcano)
• Hands-on experience with LLM inference…