Accessibility navigation


Forecast verification: relating deterministic and probabilistic metrics

Leung, T. Y. ORCID: https://orcid.org/0000-0003-0056-284X, Leutbecher, M., Reich, S. and Shepherd, T. G. (2021) Forecast verification: relating deterministic and probabilistic metrics. Quarterly Journal of the Royal Meteorological Society, 147 (739). pp. 3124-3134. ISSN 1477-870X

[img]
Preview
Text (Open access) - Published Version
· Available under License Creative Commons Attribution.
· Please see our End User Agreement before downloading.

1MB
[img] Text (Open access) - Published Version
· Restricted to Repository staff only
· Available under License Creative Commons Attribution.

1MB
[img] Text - Accepted Version
· Restricted to Repository staff only

1MB

It is advisable to refer to the publisher's version if you intend to cite from this work. See Guidance on citing.

To link to this item DOI: 10.1002/qj.4120

Abstract/Summary

The philosophy of forecast verification is rather different between deterministic and probabilistic verification metrics: generally speaking, deterministic metrics measure differences, whereas probabilistic metrics assess reliability and sharpness of predictive distributions. This paper considers the root-mean-square error (RMSE), which can be seen as a deterministic metric, and the probabilistic metric Continuous Ranked Probability Score (CRPS), and demonstrates that under certain conditions, the CRPS can be mathematically expressed in terms of the RMSE when these metrics are aggregated. One of the required conditions is the normality of distributions. The other condition is that, while the forecast ensemble need not be calibrated, any bias or over- / under-dispersion cannot depend on the forecast distribution itself. Under these conditions, the CRPS is a fraction of the RMSE, and this fraction depends only on the heteroscedasticity of the ensemble spread and the measures of calibration. The derived CRPS-RMSE relationship for the case of perfect ensemble reliability is tested on simulations of idealised two-dimensional barotropic turbulence. Results suggest that the relationship holds approximately despite that the normality condition is not met.

Item Type:Article
Refereed:Yes
Divisions:Science > School of Mathematical, Physical and Computational Sciences > Department of Mathematics and Statistics
Science > School of Mathematical, Physical and Computational Sciences > Department of Meteorology
ID Code:99066
Publisher:Wiley

Downloads

Downloads per month over past year

University Staff: Request a correction | Centaur Editors: Update this record

Page navigation