Accessibility navigation

Principles for automated and reproducible benchmarking

Koskela, T. ORCID:, Christidi, I. ORCID:, Giordano, M. ORCID:, Dubrovska, E. ORCID:, Quinn, J. ORCID:, Maynard, C. ORCID:, Case, D. ORCID:, Olgu, K. ORCID: and Deakin, T. ORCID: (2023) Principles for automated and reproducible benchmarking. In: SC-W 2023: Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis, 12-17 Nov 2023, Denver, Colorado, pp. 609-618, (ISBN: 9798400707858)

Text (Open Access) - Published Version
· Available under License Creative Commons Attribution Non-commercial.
· Please see our End User Agreement before downloading.


It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1145/3624062.3624133


The diversity in processor technology used by High Performance Computing (HPC) facilities is growing, and so applications must be written in such a way that they can attain high levels of performance across a range of different CPUs, GPUs, and other accelerators. Measuring application performance across this wide range of platforms becomes crucial, but there are significant challenges to do this rigorously, in a time efficient way, whilst assuring results are scientifically meaningful, reproducible, and actionable. This paper presents a methodology for measuring and analysing the performance portability of a parallel application and shares a software framework which combines and extends adopted technologies to provide a usable benchmarking tool. We demonstrate the flexibility and effectiveness of the methodology and benchmarking framework by showcasing a variety of benchmarking case studies which utilise a stable of supercomputing resources at a national scale.

Item Type:Conference or Workshop Item (Paper)
Divisions:Science > School of Mathematical, Physical and Computational Sciences > Department of Computer Science
Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
ID Code:114121


Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation