HPC Infrastructure & Cluster Engineer- TS​/SCI Cleared

HPC Infrastructure & Cluster Engineer

Springfield, VA

Marathon TS is seeking a TS/SCI-cleared Infrastructure Engineer to manage and optimize a high-performance compute environment supporting intensive AI/ML workloads. This role is heavily focused on Linux cluster administration, GPU infrastructure, workload scheduling, high-speed networking, storage, and containerized environments.

Key Responsibilities:

• Cluster Administration:
  Manage the day-to-day operations of the customer compute cluster, including Linux operating system administration, hardware monitoring, patching, and system upgrades.
• Resource and Job Management:
  Configure, maintain, and optimize workload management and orchestration platforms, utilizing the Run:
  
  AI job scheduler to ensure efficient distribution of intensive AI/ML workloads across the cluster.
• Infrastructure Optimization:
  Tune cluster performance at the hardware, operating system, and network levels to maximize compute efficiency and data throughput for customer workloads.
• Storage and Network Management:
  Administer storage solutions and high-speed networking fabrics. Support the transition to and ongoing management of an Infini Band GPU-to-GPU network infrastructure to minimize latency for distributed operations.
• Environment Configuration:
  Partner with technology integration teams to provision specific environments, dependencies, and container platforms, specifically leveraging Red Hat Open Shift, required for seamless customer model deployment.
• Security and Compliance:
  Ensure all infrastructure components remain compliant with federal security standards, implementing strict access controls and maintaining system accreditations.

Basic Qualifications:

• Clearance:
  Active TS/SCI with the ability to obtain CI Poly.
• Experience:
  
  5+ years of experience in Linux systems administration and infrastructure management with a specific focus on high-performance computing environments.
• Technical
  
  Skills:
• Expertise in managing bare-metal servers, enterprise storage arrays, and advanced network configurations (Experience with Infini Band).
• Strong proficiency with workload managers, job schedulers, and AI orchestration tools (e.g., Run:AI, SLURM).
• Hands-on experience with enterprise container orchestration platforms, specifically Open Shift or Kubernetes.
• Experience writing automation and configuration scripts (e.g., Bash, Python) to streamline cluster maintenance.
• Troubleshooting Focus:
  Proven ability to diagnose and resolve complex hardware, network, and OS-level issues.

Preferred Qualifications:

• Familiarity with parallel file systems and high-throughput storage architectures.
• Prior experience engineering or managing high-speed GPU-to-GPU communication topologies.

US Citizenship Required

TS/SCI Required