HPC System Security Engineer

** Department
* * Provost Research Computing Center
** About the Department
** The University of Chicago Research Computing Center (RCC), a unit in the Office of Research, provides high-end research computing resources to researchers at the University of Chicago. It is dedicated to enabling research by providing access to centrally managed High-Performance Computing (HPC), storage, and visualization resources. These resources include hardware, software, high-level scientific and technical user support, and the education and training required to help researchers make full use of modern HPC technology and local and national supercomputing resources.

The Office of Research oversees the conduct of sponsored research, research program development, and contract management functions.
** Job Summary
** The job participates in the design of automated, scalable, and rapidly deployable solutions to systems infrastructure and server configuration. Installs, configures, and maintains operating systems, monitoring and alerting systems, utility software, and firewalls. Plans and executes hands-on maintenance for production servers as well as Windows and Linux servers.
The University of Chicago is seeking a highly qualified HPC Systems Security Engineer to join the HPC Systems and Operations team that builds and manages RCC's HPC infrastructure. The individual in this position will be involved in the operation, maintenance, security, and compliance of large-scale complex HPC systems primarily used for research.
** Responsibilities
* ** Design, deploy, configure, and administer HPC clusters, including management and compute nodes, storage infrastructure, interconnects (e.g., Infini Band), and related systems.
* Develop, maintain, and enforce security procedures and system documentation for operational and compliance purposes.
* Implement infrastructure and security monitoring and detection systems to identify failures, unusual activity and respond to automated alerts.
* Tune, secure, and maintain the HPC job scheduling environment, including fair-sharing, accounting, and policy enforcement.
* Troubleshoot and resolve operational, performance, and security-related issues across HPC hardware and software stacks. Coordinate with hardware and software vendors to address defects, vulnerabilities, and performance issues. Assist Computational Scientists team with user support and helpdesk tickets, including elevated support for security-protected environments.
* Implement and maintain secure and reliable backup, archival, disaster-recovery, and restore capabilities for systems and research data.
* Perform vulnerability scanning, patch management, system and firmware updates across the infrastructure.
* Maintain complex system and network administration functions. Works with moderated guidance to administer simple systems and assists in the administration of larger systems.
* Maintains all supporting documentation for comprehensive operating system, hardware and software configuration. Monitors primary responses for information technology related security incidents and violations. Keeps current with new security and network monitoring technologies, applicable laws and regulations.
* Performs other related work as needed.
** Minimum Qualifications
***
* Education:

** Minimum requirements include a college or university degree in related field.
*
* Work Experience:

** Minimum requirements include knowledge and skills developed through 2-5 years of work experience in a related job discipline.
*
* Certifications:

****--
- **** Preferred Qualifications
***
* Experience:

*** Linux system administration experience in a large, distributed computing environment.
* Demonstrated experience and knowledge of system security and best practices.
** Technical Skills or Knowledge:
*** Knowledge of Linux administration required, RHEL.
* Experience and advanced skills in scripting with Python or Bash.
* Experience installing, configuring, and managing job schedulers (e.g., Slurm, Torque, PBS, LSF).
* Experience with automation tools such as Ansible, Puppet, Chef, Salt.
* Experience with provisioning tools (e.g., xCAT, Confluent, Warewulf).
* Experience implementing monitoring tools (e.g., Check

MK, Zabbix, Nagios).
* Knowledge of frameworks and federal regulations to protect regulated systems and data (e.g., HIPAA, FISMA, NIST CSF).
* Experience working, documenting and enforcing controls required to protect controlled unclassified information (e.g., NIST 800-53, NIST 800-171, NIST SP 800-223, FIPS).
* Knowledge of at least one distributed storage system (e.g., Storage Scale, Lustre, Gluster, BeeGFS, Ceph) and practical experience.
* Experience with Infini Band (must at least be able to demonstrate a working knowledge of concepts)
* Experience in writing precise and concise documentation, standard operating procedures.
** Preferred Competencies
*** Understand and translate researchers' scientific goals into computational requirements.
* Work well with faculty and researchers.
* Identify and gain…