On Model Selection, Bayesian Networks, and the Fisher Information Integral

Yuan Zou, Teemu Roos

Research output: Contribution to journalArticleScientificpeer-review


We study BIC-like model selection criteria and in particular, their refinements that include a constant term involving the Fisher information matrix. We perform numerical simulations that enable increasingly accurate approximation of this constant in the case of Bayesian networks. We observe that for complex Bayesian network models, the constant term is a negative number with a very large absolute value that dominates the other terms for small and moderate sample sizes. For networks with a fixed number of parameters, d, the leading term in the complexity penalty, which is proportional to d, is the same. However, as we show, the constant term can vary significantly depending on the network structure even if the number of parameters is fixed. Based on our experiments, we conjecture that the distribution of the nodes’ outdegree is a key factor. Furthermore, we demonstrate that the constant term can have a dramatic effect on model selection performance for small sample sizes.
Original languageEnglish
JournalNew Generation Computing
Issue number1
Pages (from-to)5-27
Number of pages23
Publication statusPublished - Jan 2017
MoE publication typeA1 Journal article-refereed

Fields of Science

  • 112 Statistics and probability
  • Fisher information integral
  • Bayesian networks
  • normalized maximum likelihood
  • Model selection
  • BIC
  • 113 Computer and information sciences

Cite this