HPC Systems Administrator; Hardware & Infrastructure Operations

The Sherlock HPC cluster is the flagship of Stanford’s research computing environment, supporting thousands of users and a massive variety of scientific workloads. We are looking for an HPC Systems Administrator who thrives at the intersection of high-density hardware and Linux systems engineering.

In this role, you will be the primary steward of the physical infrastructure on Sherlock and other platforms. You will ensure that our 1,500+ compute nodes, high-density GPU racks, and petabyte-scale storage arrays are meticulously maintained, expertly tuned, and highly available.

You won't just be swapping parts; you will be managing the physical backbone of a world‑class research environment. From debugging errors on NVIDIA H200s to optimizing Infini Band cabling for our Lustre scratch tiers, your work is the foundation upon which Nobel‑caliber research is built.

• Hardware Lifecycle & Deployment: Lead the physical deployment, burn‑in, troubleshooting, and de‑commissioning of compute nodes, GPU servers, and high‑density storage systems.
• Diagnostics & Root Cause Analysis: Perform troubleshooting on hardware issues—such as memory errors, GPU thermal throttling, network failures—and coordinate with vendors for support and replacements.
• Data Center Operations: Collaborate with the data center team to plan and manage hardware deployments.
• Provisioning & Automation: Work with lead platform administrators on testing and provisioning to ensure rapid, consistent deployment of cluster images across the fleet.
• Health & Telemetry: Refine hardware‑level monitoring to proactively identify failing components before they impact active research jobs.

• Education:
  
  Bachelor's degree and eight years of relevant experience, or a combination of education and relevant experience.
• Experience: 3-5+ years of experience in Linux Systems Administration, with a strong preference for candidates from HPC, large‑scale data center, or research environments.
• Hardware Proficiency: Solid understanding of x86 server architecture, GPU systems, Ethernet, and high‑performance interconnects.
• Scripting: Proficiency in scripting languages for automating hardware health checks, log parsing, and routine maintenance tasks.
• Infrastructure Management: Experience using configuration management tools to manage hardware settings and firmware versions at scale; experience working with data center teams to populate and maintain DCIM solutions preferred.
• Physical Requirements: Ability to lift up to 50 lbs and work comfortably in a data center environment, including racking equipment and managing complex cable topologies.
• Communication: Strong written and verbal communication skills.

• Direct experience maintaining hardware for HPC systems and large scale storage systems.
• Familiarity with the Slurm workload manager and how hardware health impacts job scheduling.
• Exposure to liquid cooling solutions or high‑density rack power management.

• Constantly perform desk‑based computer tasks.
• Frequently sit, grasp lightly/fine manipulation.
• Occasionally stand/walk, writing by hand.
• Rarely use a telephone, lift/carry/push/pull objects that weigh up to 10 pounds.

Consistent with its obligations under the law, the University will provide reasonable accommodations to applicants and employees with disabilities. Applicants requiring a reasonable accommodation for any part of the application or hiring process should contact Stanford University Human Resources by submitting a contact form.

• May work extended hours, evenings, and weekends.

• Interpersonal
  
  Skills:
  
  Demonstrates the ability to work well with Stanford colleagues and clients and with external organizations.
• Promote Culture of Safety:
  Demonstrates commitment to personal responsibility and value for safety; communicates safety concerns; uses and promotes safe behaviors based on training and lessons learned.
• Subject to and expected to stay in sync with all applicable University policies and procedures, including but not limited to the personnel policies and other policies found in Stanford's…