The ESGF Virtual Aggregation (CMIP6 v20240125)

Cimadevilla, E., Lawrence, B. ORCID: https://orcid.org/0000-0001-9262-7860 and Santiago Cofiño, A. (2025) The ESGF Virtual Aggregation (CMIP6 v20240125). Geoscientific Model Development, 18 (8). pp. 2461-2478. ISSN 1991-9603 doi: 10.5194/gmd-18-2461-2025

Abstract/Summary

he Earth System Grid Federation (ESGF) holds several petabytes of climate data distributed across millions of files held in data centres worldwide. The processes of ob- taining and manipulating the scientific information (climate variables) held in these files are non-trivial. The ESGF Vir- tual Aggregation is one of several solutions to provide an out-of-the-box aggregated and analysis-ready view of those variables. Here, we discuss the ESGF Virtual Aggregation in the context of the existing infrastructure and some of those other solutions providing analysis-ready data. We describe how it is constructed, how it can be used, and its benefits for model evaluation data analysis tasks, and we provide some performance evaluation. It will be seen that the ESGF Vir- tual Aggregation provides a sustainable solution to some of the problems encountered in producing analysis-ready data without the cost of data replication to different formats, al- beit at the cost of more data movement within the analysis compared to some alternatives. If heavily used, it may also require more ESGF data servers than are currently deployed in data node deployments. The need for such data servers should be a component of ongoing discussions about the fu- ture of the ESGF and its constituent core services.

Item Type	Article
URI	https://centaur.reading.ac.uk/id/eprint/127555
Identification Number/DOI	10.5194/gmd-18-2461-2025
Refereed	Yes
Divisions	Science > School of Mathematical, Physical and Computational Sciences > NCAS Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
Publisher	European Geosciences Union
Download/View statistics	View download statistics for this item