Accessibility navigation


Towards green scientific data compression through high-level I/O interfaces

Alforov, Y., Novikova, A., Kuhn, M., Kunkel, J. and Ludwig, T. (2018) Towards green scientific data compression through high-level I/O interfaces. In: SBAC-PAD 2018, 24-27 September, Lyon, France, pp. 209-216.

[img]
Preview
Text - Accepted Version
· Please see our End User Agreement before downloading.

411kB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

Official URL: https://ieeexplore.ieee.org/document/8645921

Abstract/Summary

Every HPC system today has to cope with a deluge of data generated by scientific applications, simulations or large- scale experiments. The upscaling of supercomputer systems and infrastructures, generally results in a dramatic increase of their energy consumption. In this paper, we argue that techniques like data compression can lead to significant gains in terms of power efficiency by reducing both network and storage requirements. To that end, we propose a novel methodology for achieving on-the-fly intelligent determination of energy efficient data reduction for a given data set by leveraging state-of-the-art compression algorithms and meta data at application-level I/O. We motivate our work by analyzing the energy and storage saving needs of real-life scientific HPC applications, and review the various compression techniques that can be applied. We find that the resulting data reduction can decrease the data volume transferred and stored by as much as 80% in some cases, consequently leading to significant savings in storage and networking costs.

Item Type:Conference or Workshop Item (Paper)
Refereed:Yes
Divisions:Faculty of Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
ID Code:79584
Publisher:Springer

Downloads

Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation