The idea that “simplicity is a sign of truth”, and the related “Occam’s razor” principle, stating that, all other things being equal, simpler models should be preferred to more complex ones, have been long discussed in philosophy and science. We explore these ideas in the context of supervised machine learning, namely the branch of artificial intelligence that studies algorithms which balance simplicity and accuracy in order to effectively learn about the features of the underlying domain. Focusing on statistical learning theory, we show that situations exist for which a preference for simpler models (as modeled through the addition of a regularization term in the learning problem) provably slows down, instead of favoring, the supervised learning process. Our results shed new light on the relations between simplicity and truth approximation, which are briefly discussed in the context of both machine learning and the philosophy of science.
The so-called Preface Paradox seems to show that one can rationally believe two logically incompatible propositions. We address this puzzle, relying on the notions of truthlikeness and approximate truth as studied within the post-Popperian research programme on verisimilitude. In particular, we show that adequately combining probability, approximate truth, and truthlikeness leads to an explanation of how rational belief is possible in the face of the Preface Paradox. We argue that our account is superior to other solutions of the paradox, including a recent one advanced by Hannes Leitgeb (Analysis 74.1).