### Abstract

To compare learning algorithms that differ by the adopted statistical paradigm, model class, or search heuristic, it is common to evaluate the performance on training data of varying size. Measuring the performance is straightforward if the data are generated from a known model, the ground truth. However, when the study concerns real-world data, the current methodology is limited to estimating predictive performance, typically by cross-validation. This work introduces a method to compare algorithms’ ability to learn the model structure, assuming no ground truth is given. The idea is to identify a partial structure on which the algorithms agree, and measure the performance in relation to that structure on subsamples of the data. The method is instantiated to structure learning in Bayesian networks, measuring the performance by the structural Hamming distance. It is tested using benchmark ground truth networks and algorithms that maximize various scoring functions. The results show that the method can produce evaluation outcomes that are close to those one would obtain if the ground truth was available.

Original language | English |
---|---|

Journal | Proceedings of Machine Learning Research |

Volume | 84 |

Pages (from-to) | 1570-1578 |

Number of pages | 9 |

ISSN | 1938-7228 |

Publication status | Published - 2018 |

MoE publication type | A4 Article in conference proceedings |

Event | International Conference on Artificial Intelligence and Statistics - Play Blanca, Lanzarote, Canary Islands, Spain Duration: 9 Apr 2018 → 11 Apr 2018 Conference number: 21 |

### Fields of Science

- 113 Computer and information sciences

## Cite this

Viinikka, J., Eggeling, R., & Koivisto, M. (2018). Intersection-Validation: A Method for Evaluating Structure Learning without Ground Truth.

*Proceedings of Machine Learning Research*,*84*, 1570-1578.