Presentation
Emerging Filesystem Technologies and Use Cases for AI and HPC
BoF Organizer/Speakers
Event Type
Birds of a Feather
AI/Machine Learning/Deep Learning
Big Data Analytics
Exascale Systems
File Systems
Storage Technologies
TimeTuesday, June 18th10:30am - 11:30am
LocationKontrast
DescriptionThe goal of this BOF is to engage system managers, system administrators, users, and
other interested parties in a discussion of technologies and information exchange on
best practices and lessons learned for implementing emerging, flash-based storage and
file-system technologies within HPC systems. Many universities, corporations and
research centers operate HPC clusters using a conventional architecture of
interconnected compute nodes with minimal on-node storage coupled to a hard-drive-
based (or perhaps, flash-based) parallel file system. Over the past decade, the
workload on many systems has broadened from traditional modeling & simulation tasks
to data-intensive analytics, machine learning, and AI. Often these new workloads exhibit
I/O patterns that are not well served by the traditional parallel file system architectures.
The declining cost curve of flash storage technologies, along with the emergence of new
standards such as NVMe, and new implementation architectures such as hyper-
converged file systems, offer the opportunity to incorporate new I/O subsystems or tiers
featuring high performance random access and small block/small file I/O. This BOF will
bring together interested parties in an active, discussion-oriented format to exchange
ideas, experiences and share knowledge on the deployment of these new technologies
in high performance computing and AI environments.
other interested parties in a discussion of technologies and information exchange on
best practices and lessons learned for implementing emerging, flash-based storage and
file-system technologies within HPC systems. Many universities, corporations and
research centers operate HPC clusters using a conventional architecture of
interconnected compute nodes with minimal on-node storage coupled to a hard-drive-
based (or perhaps, flash-based) parallel file system. Over the past decade, the
workload on many systems has broadened from traditional modeling & simulation tasks
to data-intensive analytics, machine learning, and AI. Often these new workloads exhibit
I/O patterns that are not well served by the traditional parallel file system architectures.
The declining cost curve of flash storage technologies, along with the emergence of new
standards such as NVMe, and new implementation architectures such as hyper-
converged file systems, offer the opportunity to incorporate new I/O subsystems or tiers
featuring high performance random access and small block/small file I/O. This BOF will
bring together interested parties in an active, discussion-oriented format to exchange
ideas, experiences and share knowledge on the deployment of these new technologies
in high performance computing and AI environments.
BoF Organizer/Speakers
Director of Industry Relations at SDSC