Abstract/Summary

Metrics are an objective, quantitative assessment of forecast (or model) agreement with observations. They are essential for assessing forecast accuracy and reliability, and consequently act as a diagnostic for forecast development. Partly as a result of limited spatial sampling of observations, much of space‐weather forecasting is focused on the time domain, rather than inherent spatial variability. Thus metrics are primarily “point‐by‐point” approaches, in which observed conditions at time t are compared directly (and only) with the forecast conditions at time t. Such metrics are undoubtedly useful. But in lacking an explicit consideration of timing uncertainties, they have limitations as diagnostic tools and can, under certain conditions, be misleading. Using a near‐Earth solar wind speed forecast as an illustrative example, this study briefly reviews the most commonly‐used point‐by‐point metrics and advocates for complementary “time window” approaches. In particular, a scale‐selective approach, originally developed in numerical weather prediction for validation of spatially patchy rainfall forecasts, is adapted to the time domain for space‐weather purposes. This simple approach readily determines the time scales over which a forecast is and isn’t valuable, allowing the results of point‐by‐point metrics to be put in greater context.