JUNE 18–22, 2017
FRANKFURT AM MAIN, GERMANY

Presentation Details
 
Name: (RP15) Decoupling the Selection of Compression Algorithms from Required Precision with the Scientific Compression Library (SCIL)
 
Time: Tuesday, June 20, 2017
08:35 am - 09:45 am
 
Room:   Substanz 1+2  
 
Breaks:07:30 am - 10:00 am Welcome Coffee
 
Presenter:   Julian Kunkel, DKRZ
 
Abstract:  
Data intense scientific domains use data compression to reduce the storage space needed. Lossless data compression preserves the original information accurately but usually yields a compression ratio of 2:1. Lossy data compression can achieve much higher compression rates depending on the tolerable error/precision needed. Therefore, the field of lossy compression is still subject to active research. From the perspective of a scientist, the actual algorithm does not matter but the qualitative information about the implied loss of precision of data is the main concern. With the Scientific Compression Library (SCIL), we are developing a meta-compressor that allows users to set various quantities that define the acceptable error and the expected performance behavior. The library then chooses the appropriate chain of algorithms to yield the users requirements. This approach is a crucial step towards a scientifically safe use of much-needed lossy data compression, because it disentangles the tasks of determining scientific ground characteristics of tolerable noise, from the task of determining an optimal compression strategy given target noise levels and constraints. Without changing applications, it allows these codes to utilize future algorithms once they are integrated into the library.

Authors:
Julian Kunkel, DKRZ
Eugen Betke, DKRZ
Anastasiia Novikova, Universität Hamburg
Armin Schaare, Universität Hamburg
  
 
Download

RP15_Kunkel.pdf (2250 KB)