Abstract/Summary

Data Assimilation means to find a trajectory of a dynamical model that matches a given set of observations. A problem of data assimilation experiments is that there is no possibility of replication. This is due to the fact that truly 'out-of-sample' observations from the same underlying flow pattern but with independent errors are usually not available. A direct evaluation against the available observations is likely to yield optimistic results since the observations were already used to find the solution. A possible remedy is presented which simply consists of estimating the optimism, giving a more realistic picture of the out-of-sample performance. The approach is simple when applied to data assimilation algorithms employing linear error feedback. Moreover, the simplicity of this method allows the optimism to be calculated in operational settings. In addition to providing a more accurate picture of performance, this approach provides a simple and efficient means to determine the optimal feedback gain matrix. A key feature of data assimilation schemes which employ linear error feedback, is the feedback gain matrix used to couple the underlying dynamical system to the assimilating algorithm. A persistent problem in practice is to find a suitable feedback. Striking the right balance of coupling strength requires a reliable assessment of performance which is provided by our estimate of the out-of-sample error. Numerical and theoretical results demonstrate that in linear systems with gaussian perturbations, the feedback determined in this way will approach the optimal Kalman Gain in the limit of large observational windows.