Accessibility navigation

An MPI-IO In-Memory driver for non-volatile pooled memory of the Kove XPD

Kunkel, J. and Betke, E. (2017) An MPI-IO In-Memory driver for non-volatile pooled memory of the Kove XPD. In: Kunkel, J., Yokota, R., Taufer, M. and Shalf, J. (eds.) High Performance Computing. Lecture Notes in Computer Science, 10524 (10524). Springer, Cham, pp. 679-690. ISBN 9783319676296

Text - Accepted Version
· Please see our End User Agreement before downloading.


It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1007/978-3-319-67630-2_48


Many scientific applications are limited by the performance offered by parallel file systems. SSD based burst buffers provide significant better performance than HDD backed storage but at the expense of capacity. Clearly, achieving wire-speed of the interconnect and predictable low latency I/O is the holy grail of storage. In-memory storage promises to provide optimal performance exceeding SSD based solutions. Kove®’s XPD® offers pooled memory for cluster systems. This remote memory is asynchronously backed up to storage devices of the XPDs and considered to be non-volatile. Albeit the system offers various APIs to access this memory such as treating it as a block device, it does not allow to expose it as file system that offers POSIX or MPI-IO semantics. In this paper, we (1) describe the XPD-MPIIO-driver which supports the scale-out architecture of the XPDs. This MPI-agnostic driver enables high-level libraries to utilize the XPD’s memory as storage. (2) A thorough performance evaluation of the XPD is conducted. This includes scale-out testing of the infrastructure and “metadata” operations but also performance variability. We show that the driver and storage architecture is able to nearly saturate wire-speed of Infiniband (60+ GiB/s with 14FDR links) while providing low latency and little performance variability.

Item Type:Book or Report Section
Divisions:Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
ID Code:77670


Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation