Inductive reasoning and Kolmogorov complexity

AbstractReasoning to obtain the “truth” about reality, from external data, is an important, controversial, and complicated issue in man's effort to understand nature. (Yet, today, we try to make machines do this.) There have been old useful principles, new exciting models, and intricate theories scattered in vastly different areas including philosophy of science, statistics, computer science, and psychology. We focus on inductive reasoning in correspondence with ideas of R. J. Solomonoff. While his proposals result in perfect procedures, they involve the noncomputable notion of Kolmogorov complexity. In this paper we develop the thesis that Solomonoff's method is fundamental in the sense that many other induction principles can be viewed as particular ways to obtain computable approximations to it. We demonstrate this explicitly in the cases of Gold's paradigm for inductive inference, Rissanen's minimum description length (MDL) principle, Fisher's maximum likelihood principle, and Jaynes' maximum entropy principle. We present several new theorems and derivations to this effect. We also delimit what can be learned and what cannot be learned in terms of Kolmogorov complexity, and we describe an experiment in machine learning of handwritten characters. We also give an application of Kolmogorov complexity in Valiant style learning, where we want to learn a concept probably approximately correct in feasible time and examples

A practical Bayesian framework for backpropagation networks

A quantitative and practical Bayesian framework is described for learning of mappings in feedforward networks. The framework makes possible (1) objective comparisons between solutions using alternative network architectures, (2) objective stopping rules for network pruning or growing procedures, (3) objective choice of magnitude and type of weight decay terms or additive regularizers (for penalizing large weights, etc.), (4) a measure of the effective number of well-determined parameters in a model, (5) quantified estimates of the error bars on network parameters and on network output, and (6) objective comparisons with alternative learning and interpolation models such as splines and radial basis functions. The Bayesian "evidence" automatically embodies "Occam's razor," penalizing overflexible and overcomplex models. The Bayesian approach helps detect poor underlying assumptions in learning models. For learning models well matched to a problem, a good correlation between generalization ability and the Bayesian evidence is obtained

Predicting Defection

Eric Posner\u27s cooperation theory of social norms develops from rational choice theory an austere and powerful explanation of why people comply with social norms. He illustrates his theory with subtle analysis of a number of legal issues. The book will help anyone influenced by law and economics to incorporate into her thinking the work in sociology, psychology, and ethics that bears on human behavior. Most readers will find applications for Posner\u27s theory

Predicting Defection

Eric Posner\u27s cooperation theory of social norms develops from rational choice theory an austere and powerful explanation of why people comply with social norms. He illustrates his theory with subtle analysis of a number of legal issues. The book will help anyone influenced by law and economics to incorporate into her thinking the work in sociology, psychology, and ethics that bears on human behavior. Most readers will find applications for Posner\u27s theory

The Proton and Occam's Razor

Otto Stern's 1933 measurement of the unexpectedly large proton magnetic moment indicated to most physicists that the proton is not a point particle. At that time, many physicists modeled elementary particles as point particles, and therefore Stern's discovery initiated the speculation that the proton might be a composite particle. In this work, we show that despite being an elementary particle, the proton is an extended particle. Our work is motivated by the experimental data, which we review in section 1. By applying Occam's Razor principle, we identify a simple proton structure that explains the origin of its principal parameters. Our model uses only relativistic and electromagnetic concepts, highlighting the primary role of the electromagnetic potentials and of the magnetic flux quantum ΦM = h/e. Unlike prior proton models, our methodology does not violate Maxwell's equation, Noether's theorem, or the Pauli exclusion principle. Considering that the proton has an anapole (toroidal) magnetic moment, we propose that the proton is a spherical shaped charge that moves at the speed of light along a path that encloses a toroidal volume. A magnetic flux quantum ΦM = h/e stabilizes the proton's charge trajectory. The two curvatures of the toroidal and poloidal current loops are determined by the magnetic forces associated with ΦM. We compare our calculations against experimental data

Approaches to abductive reasoning : an overview

Abduction is a form of non-monotonic reasoning that has gained increasing interest in the last few years. The key idea behind it can be represented by the following inference rule frac{varphirightarrowomega,}{varphi}omega, i.e., from an occurrence of omega and the rule "varphi implies omega';, infer an occurrence of varphi as a plausible hypothesis or explanation for omega. Thus, in contrast to deduction, abduction is as well as induction a form of "defeasible'; inference, i.e., the formulae sanctioned are plausible and submitted to verification. In this paper, a formal description of current approaches is given. The underlying reasoning process is treated independently and divided into two parts. This includes a description of methods for hypotheses generation and methods for finding the best explanations among a set of possible ones. Furthermore, the complexity of the abductive task is surveyed in connection with its relationship to default reasoning. We conclude with the presentation of applications of the discussed approaches focusing on plan recognition and plan generation