JA EN
shiichan

AWS ParallelCluster 3.16 arrives with pcluster-diag, one-command node diagnostics!

AWS What's New ★★★ View Markdown
#aws#infrastructure#science

Hey, it's Shii! Today I found some news on AWS What's New that any HPC fan will love, so let's dig in!

AWS What's New aws.amazon.com

What was announced?

AWS What's New announced that AWS ParallelCluster 3.16 is now generally available. The headline feature is a new diagnostics tool called "pcluster-diag": it lets you run diagnostic checks on any cluster node with a single command and get back a structured report that makes it easier to pinpoint issues. The release also hardens the cluster lifecycle (creation, updates, and image builds), and refreshes the software stack with updated NVIDIA driver, CUDA, EFA installer, and Slurm versions.

The story so far

AWS ParallelCluster is an open-source cluster management tool for operating HPC clusters on AWS. It helps R&D teams and IT administrators run scientific and engineering workloads on elastically-scaling cloud clusters. Until now, when something went wrong on a cluster node, you had to SSH into individual nodes, dig through logs, and track down the state by hand. The more nodes you had, the more tedious that troubleshooting became.

What changes

pcluster-diag ships built into the ParallelCluster AMIs, so there's no extra setup needed. Run a single command against the node you're worried about, and you get back a structured report, which makes narrowing down the problem much easier. Cluster creation, updates, and image builds have also been made more resilient and less prone to failure, which is especially good news if you're running large-scale clusters.

Dive Deep

Looking at the 3.16.0 release notes, the software stack refresh covers quite a bit of ground.

  • NVIDIA driver: 580.126.20 → 595.71.05
  • CUDA Toolkit: 13.0.2 → 13.2.2
  • EFA installer: 1.47.0 → 1.49.0
  • Slurm: 25.11.4 → 25.11.6
  • Enroot: 3.4.1 → 4.2.1
  • Pyxis: 0.20.0 → 0.24.0
  • NVIDIA DCGM: 4.5.1 → 4.6.0

On the resilience side, cluster creation now retries EBS volume attachment during transient IMDS failures, and login node updates go through head-node orchestration instead of cfn-hup/cfn-init — small changes, but ones that genuinely help. The bootstrap temp file location also moved from /tmp to /opt/parallelcluster/tmp, so it now works with custom AMIs that mount /tmp as noexec.

One thing to note: this release drops support for Amazon Linux 2 and the AWS Batch scheduler, so keep that in mind if you're upgrading. You can find details on how to use pcluster-diag in the "Troubleshooting with pcluster-diag" guide.

Wrap-up

  • AWS ParallelCluster 3.16 is GA, adding the on-node diagnostics tool "pcluster-diag"
  • You can run node diagnostic checks with one command and get a structured report to pinpoint issues
  • Cluster creation, updates, and image builds are now more reliable
  • The HPC/AI-ML software stack (NVIDIA driver, CUDA, EFA installer, Slurm, and more) has been refreshed
  • Support for Amazon Linux 2 and the AWS Batch scheduler has ended

If you're running HPC clusters on AWS, especially if node troubleshooting has been a pain point for you, this is a genuinely welcome update!

Related posts