AI Platform Engineer, Training and Inference

AI Platform Engineer – Training & Inference

Saviynt’s AI-powered identity platform manages and governs human and non-human access toall of an organization’s applications, data, and business processes. Customers trust Saviynt to safeguard their digital assets, drive operational efficiency, and reduce compliance costs. Builtfor the AI age, Saviynt is today helping organizations safely accelerate their deployment andusage of AI. Saviynt is recognized as the leader in identity security, with solutions that protect and empower the world’s leading brands, Fortune 500 companies and government institutions.

For more information, please visit

The AI Platform team is building the compute layer that trains, evaluates, and serves every AImodel  need an ML Platform Engineer to own distributed training on Ray +H100s, the multi-engine LLM inference mesh (vLLM, SGLang, NVIDIA Triton), and the fullmodel promotion lifecycle — from shadow mode through canary rollout to GA.

The AI Platform team’s mission is to build a secure, scalable, product-agnostic AI foundationthat enables Saviynt’s identity products to deliver measurable AI-powered outcomes. Training &Inference is the engine — it turns data into deployed models that make Saviynt’s productssmarter.

• Own the Ray ecosystem end-to-end: manage Kube Ray on GKE, tune Ray Core Task/Actor scheduling, operate the Plasma distributed object store, and configure Ray Data for GPU-direct streaming from GCS/S3
• Operate distributed training with Ray Train: configure Torch Trainer + DDP/NCCL formulti-node H100 clusters, manage checkpoint lifecycle, implement spot-preemptionrecovery, and integrate warm-start fine-tuning for retrain pipelines
• Build and operate the LLM inference mesh with Ray Serve: compose vLLM(Paged Attention), SGLang (Radix Attention), and NVIDIA Triton (TensorRT/ONNX) as aunified deployment graph with Plasma zero-copy memory sharing
• Optimise inference performance: configure fractional GPU allocation, enable continuousbatching, implement per-engine autoscaling based on request queue depth, and tuneKV-cache block sizes
• Design and operate the model routing layer: capability-based, version-based, and tenant-based routing with cost-aware fallback between self-hosted SLMs and cloudLLMs
• Build RL training infrastructure: define Flyte workflows for RL pipelines (rollout, reward shaping, policy update, evaluation), integrate Ray RLlib or custom PPO/GRPO loops with Ray Train, and manage replay buffer persistence on GCS
• Operate the full model promotion lifecycle: quality gate integration tests load tests(k6) shadow mode A/B gate canary (10% 100%) with golden-signal auto-rollback
• Operate the retrain pipeline: drift detection triggers, warm-start retraining, relative qualitygates (V2 >= V1 − 2%), and automated Flyte DAG through to canary
• Integrate RAG retrieval into the inference mesh: vector similarity search, contextassembly, and prompt construction before LLM inference

• Experience in ML engineering with time in an ML platform or MLOps role
• Production Ray depth:
  Ray Train, Serve, Core, and Data — debugged real productionfailures including NCCL timeouts, Plasma OOM, and Serve autoscaling lag
• LLM serving engines: hands-on with vLLM, SGLang, or NVIDIA Triton —Paged Attention, prefix caching, and continuous batching tuned for latency/throughputtargets
• Distributed training: DDP, FSDP, NCCL collectives, gradient checkpointing, and mixedprecision (BF16/FP8)
• RL working knowledge: PPO, policy gradient, or RLHF — able to translate an algorithminto distributed compute primitives
• Model lifecycle operations: MLflow registry, shadow/A/B/canary patterns, and auto-rollback on golden signal degradation
• Vector databases:
  Pgvector or Qdrant — ANN index strategies, embedding upsert, andquery latency tuning under inference load
• Strong Python and PyTorch;
  Flyte or equivalent ML orchestrator
• Quantization (nice to have): INT8/INT4/FP8 post-training quantization (GPTQ, AWQ, orbitsandbytes)
• Bachelor’s degree in Computer Science, Engineering, or a related field, or equivalent practical experience or equivalent military experience

We offer you a competitive total rewards package,…