2 research outputs found
No Free Lunch versus Occam's Razor in Supervised Learning
The No Free Lunch theorems are often used to argue that domain specific
knowledge is required to design successful algorithms. We use algorithmic
information theory to argue the case for a universal bias allowing an algorithm
to succeed in all interesting problem domains. Additionally, we give a new
algorithm for off-line classification, inspired by Solomonoff induction, with
good performance on all structured problems under reasonable assumptions. This
includes a proof of the efficacy of the well-known heuristic of randomly
selecting training data in the hope of reducing misclassification rates.Comment: 16 LaTeX pages, 1 figur
No free lunch versus Occam's Razor in supervised learning
The No Free Lunch theorems are often used to argue that domain specific knowledge is required to design successful algorithms. We use algorithmic information theory to argue the case for a universal bias allowing an algorithm to succeed in all interestin