ML Infrastructure Engineer

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a ML Infrastructure Engineer based in United States.

This role focuses on building and operating the core platform that powers large-scale machine learning training and inference workloads.

You will work on GPU cluster infrastructure spanning cloud, on-prem, and hybrid environments.

The position plays a critical role in enabling efficient, reliable, and scalable AI development across multiple teams.

You will design systems for scheduling, distributed training, storage throughput, and high-performance networking.

The environment is highly technical, combining systems engineering, ML frameworks, and platform reliability at scale.

You will collaborate closely with ML researchers and engineers to optimize performance, cost, and developer experience.

This is a hands‑on engineering role where impact is measured by infrastructure efficiency and production readiness of AI workloads.

• Design, build, and operate GPU and accelerator infrastructure for large-scale training and inference workloads across cloud, on-prem, and hybrid environments.
• Develop scheduling, queueing, and resource management systems to maximize utilization of compute clusters.
• Integrate and support ML frameworks such as PyTorch, JAX, Deep Speed, FSDP, Megatron‑LM, and Ray-based training workflows.
• Build and maintain high-performance storage and data pipelines ensuring consistent GPU throughput.
• Design and optimize networking layers including RDMA, Infini Band, and NCCL-based communication.
• Implement observability, monitoring, and failure analysis tools for distributed ML workloads.
• Drive automation for provisioning, lifecycle management, and infrastructure configuration.
• Partner with ML teams to forecast capacity needs and improve developer workflows and tooling.
• Ensure security, isolation, and multi‑tenant access control across AI infrastructure systems.
• Optimize cost efficiency across compute, storage, and networking through intelligent resource management.

• Bachelor’s or Master’s degree in Computer Science or related field.
• 6+ years of experience in infrastructure, platform engineering, or high-performance computing environments.
• Hands‑on experience operating GPU clusters or large-scale ML training systems in production.
• Strong proficiency in Python and at least one systems programming language (Go or C++ preferred).
• Deep understanding of distributed systems, accelerator architectures, and ML training workflows.
• Experience with Kubernetes, Slurm, Ray, or similar orchestration/scheduling systems.
• Strong knowledge of Linux internals, networking concepts, and high-performance storage systems.
• Familiarity with at least one major cloud provider’s ML infrastructure stack.
• Solid software engineering practices including testing, CI/CD, and code review workflows.
• Strong communication skills and ability to collaborate across research and engineering teams.
• Experience with RDMA/Infini Band, Fin Ops for ML workloads, or open-source ML infrastructure is a plus.

• Competitive salary range: $100,000 – $150,000
• 100% remote (within the United States)
• Full-time W2 employment structure
• H1B transfer support for eligible candidates
• Opportunity to work on large-scale AI infrastructure systems
• Exposure to cutting-edge ML frameworks and distributed training technologies
• Strong engineering culture focused on performance, reliability, and scalability
• Direct impact on production AI systems and research acceleration
• Comprehensive career growth opportunities in advanced ML infrastructure.