Compute Platform Engineer

Key Responsibilities

• Designs, builds, and operates tools, services, workflows, etc that deliver high value through the solution to key business problems,
• Responsible for development of key components of a hybrid on-prem/cloud compute platform for both interactive and scalable batch computing and establishing of processes and workflows to transition existing HPC users and teams to this platform
• Responsible for code-driven environment, applications, and container/image builds as well as CI/CD driven application deployments.
• Consult science users on application scalability to PBs of data by having a deep understanding of software engineering, algorithms, and underlying hardware infrastructure and their impact on performance.
• Confidently optimizes design and execution of complex solutions within large-scale distributed computing environments
• Produces well‑engineered software, including appropriate automated test suites, technical documentation, and operational strategy
• Ensure consistent application of platform abstractions to ensure quality and consistency with respect to logging and lineage
• Fully versed in coding best practices and ways of working, and participates in code reviews and partnering to improve the team’s standards
• Adhere to QMS framework and CI/CD best practices and helps to guide improvements to them that improve ways of working

• Bachelor's degree in Data Engineering, Computer Science, Software Engineering or related field
• Experience in programming language: e.g., Python, Go, C++, Scala, Java, including tool chains for documentation, testing, and operations / observability
• Experience in modern software development tools / ways of working (e.g. git/Git Hub, devops tools, metrics / monitoring, …)
• Cloud experience (e.g., AWS, Google Cloud, Azure), including infrastructure-as-code tools and scalable compute technologies, such as Google Batch and Vertex
• Experience with CI/CD implementations using git and a common CI/CD stack (e.g., Azure Dev Ops, Cloud Build, Jenkins, Circle
  
  CI, Git Lab)
• Experience in Linux/Unix

• Experience with Docker, Kubernetes, and the larger CNCF ecosystem including experience with application deployment tools such as Helm
• Experience with low level application build tools (make, CMake) as well as automated build systems such as spack or easybuild
• Application experience of CI/CD implementations using git and a common CI/CD stack (e.g., Jenkins, Circle
  
  CI, Git Lab, Azure Dev Ops)
• Experience in workflow orchestration with tools such as Argo Workflow, Airflow, and scientific workflow tools such as Nextflow, Snakemake, Vis Trails, or Cromwell
• Experience with application performance tuning and optimization, including in parallel and distributed computing paradigms and communication libraries such as MPI, OpenMP, Gloo, including deep understanding of the underlying systems (hardware, networks, storage) and their impact on application performance.
• Experience with agile software development environments using tools like Jira and Confluence
• Familiarity with the tools, techniques, optimizations in high-performance applications space, including engagement with the opensource community (and potentially making contributions to such tools)

A Compute Platform Engineer I is a technical contributor who can consistently take a poorly defined business or technical problem, work it to a well‑defined problem / specification, and execute on it at a high level. They have a strong focus on metrics, both for the impact of their work and for its inner workings / operations. They are a model for the team on best practice for software development in general (and their specialization in particular), including code quality, documentation, Dev Ops, and testing.

They ensure robustness of our services and serve as an escalation point in the operation of existing services, pipelines, and workflows.

A Compute Platform Engineer I should be familiar with the tools of their specialization and of their customers and engaged with the open‑source community surrounding them – potentially, even to the level of contributing pull requests.