12,181 research outputs found
An Algorithmic Approach to Information and Meaning
I will survey some matters of relevance to a philosophical discussion of
information, taking into account developments in algorithmic information theory
(AIT). I will propose that meaning is deep in the sense of Bennett's logical
depth, and that algorithmic probability may provide the stability needed for a
robust algorithmic definition of meaning, one that takes into consideration the
interpretation and the recipient's own knowledge encoded in the story attached
to a message.Comment: preprint reviewed version closer to the version accepted by the
journa
Algorithmic statistics revisited
The mission of statistics is to provide adequate statistical hypotheses
(models) for observed data. But what is an "adequate" model? To answer this
question, one needs to use the notions of algorithmic information theory. It
turns out that for every data string one can naturally define
"stochasticity profile", a curve that represents a trade-off between complexity
of a model and its adequacy. This curve has four different equivalent
definitions in terms of (1)~randomness deficiency, (2)~minimal description
length, (3)~position in the lists of simple strings and (4)~Kolmogorov
complexity with decompression time bounded by busy beaver function. We present
a survey of the corresponding definitions and results relating them to each
other
Physical limits of inference
I show that physical devices that perform observation, prediction, or
recollection share an underlying mathematical structure. I call devices with
that structure "inference devices". I present a set of existence and
impossibility results concerning inference devices. These results hold
independent of the precise physical laws governing our universe. In a limited
sense, the impossibility results establish that Laplace was wrong to claim that
even in a classical, non-chaotic universe the future can be unerringly
predicted, given sufficient knowledge of the present. Alternatively, these
impossibility results can be viewed as a non-quantum mechanical "uncertainty
principle". Next I explore the close connections between the mathematics of
inference devices and of Turing Machines. In particular, the impossibility
results for inference devices are similar to the Halting theorem for TM's.
Furthermore, one can define an analog of Universal TM's (UTM's) for inference
devices. I call those analogs "strong inference devices". I use strong
inference devices to define the "inference complexity" of an inference task,
which is the analog of the Kolmogorov complexity of computing a string. However
no universe can contain more than one strong inference device. So whereas the
Kolmogorov complexity of a string is arbitrary up to specification of the UTM,
there is no such arbitrariness in the inference complexity of an inference
task. I end by discussing the philosophical implications of these results,
e.g., for whether the universe "is" a computer.Comment: 43 pages, updated version of Physica D version, which originally
appeared in 2007 CNLS conference on unconventional computatio
Normalized Information Distance
The normalized information distance is a universal distance measure for
objects of all kinds. It is based on Kolmogorov complexity and thus
uncomputable, but there are ways to utilize it. First, compression algorithms
can be used to approximate the Kolmogorov complexity if the objects have a
string representation. Second, for names and abstract concepts, page count
statistics from the World Wide Web can be used. These practical realizations of
the normalized information distance can then be applied to machine learning
tasks, expecially clustering, to perform feature-free and parameter-free data
mining. This chapter discusses the theoretical foundations of the normalized
information distance and both practical realizations. It presents numerous
examples of successful real-world applications based on these distance
measures, ranging from bioinformatics to music clustering to machine
translation.Comment: 33 pages, 12 figures, pdf, in: Normalized information distance, in:
Information Theory and Statistical Learning, Eds. M. Dehmer, F.
Emmert-Streib, Springer-Verlag, New-York, To appea
Algorithmic Randomness as Foundation of Inductive Reasoning and Artificial Intelligence
This article is a brief personal account of the past, present, and future of
algorithmic randomness, emphasizing its role in inductive inference and
artificial intelligence. It is written for a general audience interested in
science and philosophy. Intuitively, randomness is a lack of order or
predictability. If randomness is the opposite of determinism, then algorithmic
randomness is the opposite of computability. Besides many other things, these
concepts have been used to quantify Ockham's razor, solve the induction
problem, and define intelligence.Comment: 9 LaTeX page
Kolmogorov Complexity in perspective. Part II: Classification, Information Processing and Duality
We survey diverse approaches to the notion of information: from Shannon
entropy to Kolmogorov complexity. Two of the main applications of Kolmogorov
complexity are presented: randomness and classification. The survey is divided
in two parts published in a same volume. Part II is dedicated to the relation
between logic and information system, within the scope of Kolmogorov
algorithmic information theory. We present a recent application of Kolmogorov
complexity: classification using compression, an idea with provocative
implementation by authors such as Bennett, Vitanyi and Cilibrasi. This stresses
how Kolmogorov complexity, besides being a foundation to randomness, is also
related to classification. Another approach to classification is also
considered: the so-called "Google classification". It uses another original and
attractive idea which is connected to the classification using compression and
to Kolmogorov complexity from a conceptual point of view. We present and unify
these different approaches to classification in terms of Bottom-Up versus
Top-Down operational modes, of which we point the fundamental principles and
the underlying duality. We look at the way these two dual modes are used in
different approaches to information system, particularly the relational model
for database introduced by Codd in the 70's. This allows to point out diverse
forms of a fundamental duality. These operational modes are also reinterpreted
in the context of the comprehension schema of axiomatic set theory ZF. This
leads us to develop how Kolmogorov's complexity is linked to intensionality,
abstraction, classification and information system.Comment: 43 page
- …