Presentation
(RP16) HuronFS : Hierarchical, User-level and On-demand Burst Buffer File System
SessionResearch Posters Session
Event Type
Research Poster
File Systems
Storage Technologies
TimeTuesday, June 26th8:30am - 10am
LocationSubstanz 1, 2
DescriptionIn HPC centers, the performance of the parallel file systems (PFSs) cannot match the dramatic leaps in computational performance, resulting in degradation in the performance of data-intensive applications. Moreover, I/O interference from other users further exacerbates the situation. Burst buffers, a group of dedicated buffer nodes, have been proposed to alleviate such performance gap by buffering the I/O data between compute nodes and PFSs. However, integrating burst buffer into the HPC centers requires additional procurement, deployment, and maintenance. Hence it is not economically or logistically feasible to add a burst buffer system to every existing system. Moreover, physically deployed burst buffer has its designed capacity and performance, which can hardly adapt to any demand changes. In this poster, we propose a software- and user-level on-demand burst buffer system, HuronFS, to solve such challenges. HuronFS extends our previous work CloudBB by having high-performance network and data swapping supports. HuronFS leverages compute nodes to build a software-level burst buffer system. HuronFS is designed entirely on user-level. Hence HuronFS can be easily deployed on any HPC center or can even be deployed for a certain job by a user. Moreover, HuronFS supports expending and shrinking total buffer capacity according to the workload. We evaluate HuronFS on a supercomputer, the results show that we can achieve up to 7 times speed up on sequential I/O performance compare to the NFS storage by using only two additional compute nodes. The evaluation of real applications also shows that HuronFS can help to accelerate data-intensive applications.
Poster PDF