40,675 research outputs found
Minimum Description Length Induction, Bayesianism, and Kolmogorov Complexity
The relationship between the Bayesian approach and the minimum description
length approach is established. We sharpen and clarify the general modeling
principles MDL and MML, abstracted as the ideal MDL principle and defined from
Bayes's rule by means of Kolmogorov complexity. The basic condition under which
the ideal principle should be applied is encapsulated as the Fundamental
Inequality, which in broad terms states that the principle is valid when the
data are random, relative to every contemplated hypothesis and also these
hypotheses are random relative to the (universal) prior. Basically, the ideal
principle states that the prior probability associated with the hypothesis
should be given by the algorithmic universal probability, and the sum of the
log universal probability of the model plus the log of the probability of the
data given the model should be minimized. If we restrict the model class to the
finite sets then application of the ideal principle turns into Kolmogorov's
minimal sufficient statistic. In general we show that data compression is
almost always the best strategy, both in hypothesis identification and
prediction.Comment: 35 pages, Latex. Submitted IEEE Trans. Inform. Theor
Applying MDL to Learning Best Model Granularity
The Minimum Description Length (MDL) principle is solidly based on a provably
ideal method of inference using Kolmogorov complexity. We test how the theory
behaves in practice on a general problem in model selection: that of learning
the best model granularity. The performance of a model depends critically on
the granularity, for example the choice of precision of the parameters. Too
high precision generally involves modeling of accidental noise and too low
precision may lead to confusion of models that should be distinguished. This
precision is often determined ad hoc. In MDL the best model is the one that
most compresses a two-part code of the data set: this embodies ``Occam's
Razor.'' In two quite different experimental settings the theoretical value
determined using MDL coincides with the best value found experimentally. In the
first experiment the task is to recognize isolated handwritten characters in
one subject's handwriting, irrespective of size and orientation. Based on a new
modification of elastic matching, using multiple prototypes per character, the
optimal prediction rate is predicted for the learned parameter (length of
sampling interval) considered most likely by MDL, which is shown to coincide
with the best value found experimentally. In the second experiment the task is
to model a robot arm with two degrees of freedom using a three layer
feed-forward neural network where we need to determine the number of nodes in
the hidden layer giving best modeling performance. The optimal model (the one
that extrapolizes best on unseen examples) is predicted for the number of nodes
in the hidden layer considered most likely by MDL, which again is found to
coincide with the best value found experimentally.Comment: LaTeX, 32 pages, 5 figures. Artificial Intelligence journal, To
appea
A Complete Theory of Everything (will be subjective)
Increasingly encompassing models have been suggested for our world. Theories
range from generally accepted to increasingly speculative to apparently bogus.
The progression of theories from ego- to geo- to helio-centric models to
universe and multiverse theories and beyond was accompanied by a dramatic
increase in the sizes of the postulated worlds, with humans being expelled from
their center to ever more remote and random locations. Rather than leading to a
true theory of everything, this trend faces a turning point after which the
predictive power of such theories decreases (actually to zero). Incorporating
the location and other capacities of the observer into such theories avoids
this problem and allows to distinguish meaningful from predictively meaningless
theories. This also leads to a truly complete theory of everything consisting
of a (conventional objective) theory of everything plus a (novel subjective)
observer process. The observer localization is neither based on the
controversial anthropic principle, nor has it anything to do with the
quantum-mechanical observation process. The suggested principle is extended to
more practical (partial, approximate, probabilistic, parametric) world models
(rather than theories of everything). Finally, I provide a justification of
Ockham's razor, and criticize the anthropic principle, the doomsday argument,
the no free lunch theorem, and the falsifiability dogma.Comment: 26 LaTeX page
Quantifying selection in immune receptor repertoires
The efficient recognition of pathogens by the adaptive immune system relies
on the diversity of receptors displayed at the surface of immune cells. T-cell
receptor diversity results from an initial random DNA editing process, called
VDJ recombination, followed by functional selection of cells according to the
interaction of their surface receptors with self and foreign antigenic
peptides. To quantify the effect of selection on the highly variable elements
of the receptor, we apply a probabilistic maximum likelihood approach to the
analysis of high-throughput sequence data from the -chain of human
T-cell receptors. We quantify selection factors for V and J gene choice, and
for the length and amino-acid composition of the variable region. Our approach
is necessary to disentangle the effects of selection from biases inherent in
the recombination process. Inferred selection factors differ little between
donors, or between naive and memory repertoires. The number of sequences shared
between donors is well-predicted by the model, indicating a purely stochastic
origin of such "public" sequences. We find a significant correlation between
biases induced by VDJ recombination and our inferred selection factors,
together with a reduction of diversity during selection. Both effects suggest
that natural selection acting on the recombination process has anticipated the
selection pressures experienced during somatic evolution
- …