Explanatory and Creative Alternatives

to the MDL principle

José Hernández-Orallo and Ismael García-Varea

Abstract

The Minimum Description Length (MDL) principle is the modern formalisation of Occam’s razor. It has been extensively and successfully used in machine learning (ML), especially for noisy and long sources of data. However, the MDL principle presents some paradoxes and inconven-iences. After discussing these all, we address two of the most relevant: lack of explanation and lack of creativity. We present new alternatives to address these problems. The first one, intensional complexity, avoids extensional parts in a description, so distributing compression ratio in a more even way than the MDL principle. The second one, information gain, forces that the hypothesis is informative (or computationally hard to discover) wrt. the evi-dence, so giving a formal definition of what is to discover.

Keywords: MDL Principle, Model Evaluation, Scientific and Knowledge Discovery, Occam’s Razor, Intensional Complexity, Machine Learning, Explanatory Induction, Informativeness, Creativity.

Go back to my home page .