Filter Likelihood as an Observation-Based Verification Metric in Ensemble Forecasting

Madeleine Ekblom, Lauri Tuppi, Olle Räty, Pirkka Ollinaho, Marko Laine, Heikki Järvinen

Research output: Contribution to journalArticleScientificpeer-review


In numerical weather prediction (NWP), ensemble forecasting aims to quantify the flow-dependent forecast uncertainty. The focus here is on observation-based verification of the reliability of ensemble forecasting systems. In particular, at short forecast lead times, forecast errors tend to be relatively small compared to observation errors and hence it is very important that the verification metric also accounts for observational uncertainties. This paper studies the so-called filter likelihood score which is deep-rooted in Bayesian estimation theory and fits naturally to the filtering setup of NWP. The filter likelihood score considers observation errors, ensemble mean skill, and ensemble spread in one metric. Importantly, it can be made multivariate and effortlessly expanded to simultaneous verification against all observation types through the observation operators contained in the parental data assimilation scheme. Here observations from the global radiosonde network and satellites (AMSU-A channel 5) are included in the verification of OpenIFS-based ensemble forecasts using different types of initial state perturbations. Our results show that the filter likelihood score is sensitive to the ensemble prediction system quality and compares consistently with other verification metrics such as the relationships between ensemble spread and ensemble mean forecast error, and Dawid-Sebastiani score. Our conclusion is that the filter likelihood score provides a very well-behaving verification metric, that can be made truly multivariate by including covariances, for ensemble prediction systems with a strong foundation in estimation theory.
Original languageEnglish
JournalTellus. Series A: Dynamic Meteorology and Oceanography
Issue number1
Pages (from-to)69-87
Number of pages19
Publication statusPublished - 6 Feb 2023
MoE publication typeA1 Journal article-refereed

Fields of Science

  • 119 Other natural sciences
  • Bayesian estimation
  • Kalman filtering
  • Ensemble prediction
  • Filter likelihood score
  • Numerical weather prediction
  • Probabilistic verification

Cite this