JUNE 18–22, 2017
FRANKFURT AM MAIN, GERMANY

Presentation Details
 
Name: (PP16) Advanced Data Placement via Ad-hoc File Systems
 
Time: Tuesday, June 20, 2017
03:15 pm - 03:45 pm
 
Room:   Booth #L-212  
 
Breaks:03:15 pm - 03:45 pm Coffee Break
 
Presenter:   Michael Kluge, TU Dresden
 
Abstract:   For future HPC systems, data management is an essential factor. The location of data that is needed during a calculation plays a central role in increasing the efficiency of HPC systems. While already efficient methods for reserving data exist on the processor-level, access to the parallel file system is still a bottleneck. The volume of data that is expected for the calculation within future applications exceeds the capacity of the node-local storage and requires the download of data from the parallel file system during the runtime of the application. In HPC file systems are usually a shared medium, which are used by many users in parallel. Furthermore, the performance is limited by the interface between the central file system and the compute nodes. Thus, for an application it is currently not possible to predict the actual load on the file system infrastructure and to optimize the I/O subsystem. The project aims to improve I/O performance for highly-parallel applications by distributed ad-hoc overlay file systems. For this purpose, it examines how job-specific temporary file systems can be efficiently provided for HPC environments. These file systems are to be created from the resources of the computing nodes involved. These temporary file systems are filled with the necessary data through an integration into the scheduling system of the supercomputer before the job starts. The research approach includes both the design of the file system itself as well as the questions about the proper scheduling strategy for planning the necessary I/O transfers.  

Authors: 
Michael Kluge, TU Dresden 
Sebastian Oeste, TU Dresden 
Mehmet Soysal, Karlsruhe Institut of Technology 
Marc-André Vef, University of Mainz
  
 
Download

PP16_Kluge.pdf (597 KB)