16x metadata IOPS, up to 192,000! FSx for Lustre Intelligent-Tiering gets a big boost
Hi everyone, it's Shii-chan! Today's news is a performance boost for FSx for Lustre.
AWS What's New
aws.amazon.com
What was announced?
According to AWS What's New, Amazon FSx for Lustre file systems using the Intelligent-Tiering storage class now support up to 192,000 metadata IOPS.
The story so far
The previous maximum was 12,000 IOPS. Metadata IOPS determine how many files and directories you can create, list, read, and delete per second.
What changes
- 12,000 to 192,000 IOPS, a 16x increase
- Metadata-intensive machine learning research and HPC workloads can get to results faster
- You can scale to higher performance while optimizing storage costs
Dive Deep
Per AWS, Intelligent-Tiering is the lowest-cost and only fully elastic Lustre file storage in the cloud, optimized for workloads that mix hot and cold data and don't need consistent SSD-level performance.
Availability notes:
- Applies to new Intelligent-Tiering file systems
- Available in all AWS Regions where Intelligent-Tiering is available
- See the file system metadata performance section of the FSx for Lustre User Guide
Wrap-up
- Up to 192,000 metadata IOPS (16x before)
- New Intelligent-Tiering file systems only
- Aimed at metadata-intensive ML research and HPC
This is for anyone running ML or HPC with huge numbers of files!
