Intersection-Validation: A Method for Evaluating Structure Learning without Ground Truth

Jussi Viinikka, Ralf Eggeling, Mikko Koivisto

Research output: Contribution to journalConference articleScientificpeer-review


To compare learning algorithms that differ by the adopted statistical paradigm, model class, or search heuristic, it is common to evaluate the performance on training data of varying size. Measuring the performance is straightforward if the data are generated from a known model, the ground truth. However, when the study concerns real-world data, the current methodology is limited to estimating predictive performance, typically by cross-validation. This work introduces a method to compare algorithms’ ability to learn the model structure, assuming no ground truth is given. The idea is to identify a partial structure on which the algorithms agree, and measure the performance in relation to that structure on subsamples of the data. The method is instantiated to structure learning in Bayesian networks, measuring the performance by the structural Hamming distance. It is tested using benchmark ground truth networks and algorithms that maximize various scoring functions. The results show that the method can produce evaluation outcomes that are close to those one would obtain if the ground truth was available.
Original languageEnglish
JournalProceedings of Machine Learning Research
Pages (from-to)1570-1578
Number of pages9
Publication statusPublished - 2018
MoE publication typeA4 Article in conference proceedings
EventInternational Conference on Artificial Intelligence and Statistics - Play Blanca, Lanzarote, Canary Islands, Spain
Duration: 9 Apr 201811 Apr 2018
Conference number: 21

Fields of Science

  • 113 Computer and information sciences

Cite this