8,676 research outputs found
Depth as Randomness Deficiency
Depth of an object concerns a tradeoff between computation time and excess of program length over the shortest program length required to obtain the object. It gives an unconditional lower bound on the computation time from a given program in absence of auxiliary information. Variants known as logical depth and computational depth are expressed in Kolmogorov complexity theory.
We derive quantitative relation between logical depth and computational depth and unify the different depth notions by relating them to A. Kolmogorov and L. Levin’s fruitful notion of randomness deficiency. Subsequently, we revisit the computational depth of infinite strings, study the notion of super deep sequences and relate it with other approaches
Relating and contrasting plain and prefix Kolmogorov complexity
In [3] a short proof is given that some strings have maximal plain Kolmogorov
complexity but not maximal prefix-free complexity. The proof uses Levin's
symmetry of information, Levin's formula relating plain and prefix complexity
and Gacs' theorem that complexity of complexity given the string can be high.
We argue that the proof technique and results mentioned above are useful to
simplify existing proofs and to solve open questions.
We present a short proof of Solovay's result [21] relating plain and prefix
complexity: and , (here denotes , etc.).
We show that there exist such that is infinite and is
finite, i.e. the infinitely often C-trivial reals are not the same as the
infinitely often K-trivial reals (i.e. [1,Question 1]).
Solovay showed that for infinitely many we have
and , (here
denotes the length of and , etc.). We show that this
result holds for prefixes of some 2-random sequences.
Finally, we generalize our proof technique and show that no monotone relation
exists between expectation and probability bounded randomness deficiency (i.e.
[6, Question 1]).Comment: 20 pages, 1 figur
On Algorithmic Statistics for space-bounded algorithms
Algorithmic statistics studies explanations of observed data that are good in
the algorithmic sense: an explanation should be simple i.e. should have small
Kolmogorov complexity and capture all the algorithmically discoverable
regularities in the data. However this idea can not be used in practice because
Kolmogorov complexity is not computable.
In this paper we develop algorithmic statistics using space-bounded
Kolmogorov complexity. We prove an analogue of one of the main result of
`classic' algorithmic statistics (about the connection between optimality and
randomness deficiences). The main tool of our proof is the Nisan-Wigderson
generator.Comment: accepted to CSR 2017 conferenc
Algorithmic statistics: forty years later
Algorithmic statistics has two different (and almost orthogonal) motivations.
From the philosophical point of view, it tries to formalize how the statistics
works and why some statistical models are better than others. After this notion
of a "good model" is introduced, a natural question arises: it is possible that
for some piece of data there is no good model? If yes, how often these bad
("non-stochastic") data appear "in real life"?
Another, more technical motivation comes from algorithmic information theory.
In this theory a notion of complexity of a finite object (=amount of
information in this object) is introduced; it assigns to every object some
number, called its algorithmic complexity (or Kolmogorov complexity).
Algorithmic statistic provides a more fine-grained classification: for each
finite object some curve is defined that characterizes its behavior. It turns
out that several different definitions give (approximately) the same curve.
In this survey we try to provide an exposition of the main results in the
field (including full proofs for the most important ones), as well as some
historical comments. We assume that the reader is familiar with the main
notions of algorithmic information (Kolmogorov complexity) theory.Comment: Missing proofs adde
Algorithmic statistics revisited
The mission of statistics is to provide adequate statistical hypotheses
(models) for observed data. But what is an "adequate" model? To answer this
question, one needs to use the notions of algorithmic information theory. It
turns out that for every data string one can naturally define
"stochasticity profile", a curve that represents a trade-off between complexity
of a model and its adequacy. This curve has four different equivalent
definitions in terms of (1)~randomness deficiency, (2)~minimal description
length, (3)~position in the lists of simple strings and (4)~Kolmogorov
complexity with decompression time bounded by busy beaver function. We present
a survey of the corresponding definitions and results relating them to each
other
Algorithmic statistics, prediction and machine learning
Algorithmic statistics considers the following problem: given a binary string
(e.g., some experimental data), find a "good" explanation of this data. It
uses algorithmic information theory to define formally what is a good
explanation. In this paper we extend this framework in two directions.
First, the explanations are not only interesting in themselves but also used
for prediction: we want to know what kind of data we may reasonably expect in
similar situations (repeating the same experiment). We show that some kind of
hierarchy can be constructed both in terms of algorithmic statistics and using
the notion of a priori probability, and these two approaches turn out to be
equivalent.
Second, a more realistic approach that goes back to machine learning theory,
assumes that we have not a single data string but some set of "positive
examples" that all belong to some unknown set , a property
that we want to learn. We want this set to contain all positive examples
and to be as small and simple as possible. We show how algorithmic statistic
can be extended to cover this situation.Comment: 22 page
The Thermodynamics of Network Coding, and an Algorithmic Refinement of the Principle of Maximum Entropy
The principle of maximum entropy (Maxent) is often used to obtain prior
probability distributions as a method to obtain a Gibbs measure under some
restriction giving the probability that a system will be in a certain state
compared to the rest of the elements in the distribution. Because classical
entropy-based Maxent collapses cases confounding all distinct degrees of
randomness and pseudo-randomness, here we take into consideration the
generative mechanism of the systems considered in the ensemble to separate
objects that may comply with the principle under some restriction and whose
entropy is maximal but may be generated recursively from those that are
actually algorithmically random offering a refinement to classical Maxent. We
take advantage of a causal algorithmic calculus to derive a thermodynamic-like
result based on how difficult it is to reprogram a computer code. Using the
distinction between computable and algorithmic randomness we quantify the cost
in information loss associated with reprogramming. To illustrate this we apply
the algorithmic refinement to Maxent on graphs and introduce a Maximal
Algorithmic Randomness Preferential Attachment (MARPA) Algorithm, a
generalisation over previous approaches. We discuss practical implications of
evaluation of network randomness. Our analysis provides insight in that the
reprogrammability asymmetry appears to originate from a non-monotonic
relationship to algorithmic probability. Our analysis motivates further
analysis of the origin and consequences of the aforementioned asymmetries,
reprogrammability, and computation.Comment: 30 page
A Computational Theory of Subjective Probability
In this article we demonstrate how algorithmic probability theory is applied
to situations that involve uncertainty. When people are unsure of their model
of reality, then the outcome they observe will cause them to update their
beliefs. We argue that classical probability cannot be applied in such cases,
and that subjective probability must instead be used. In Experiment 1 we show
that, when judging the probability of lottery number sequences, people apply
subjective rather than classical probability. In Experiment 2 we examine the
conjunction fallacy and demonstrate that the materials used by Tversky and
Kahneman (1983) involve model uncertainty. We then provide a formal
mathematical proof that, for every uncertain model, there exists a conjunction
of outcomes which is more subjectively probable than either of its constituents
in isolation.Comment: Maguire, P., Moser, P. Maguire, R. & Keane, M.T. (2013) "A
computational theory of subjective probability." In M. Knauff, M. Pauen, N.
Sebanz, & I. Wachsmuth (Eds.), Proceedings of the 35th Annual Conference of
the Cognitive Science Society (pp. 960-965). Austin, TX: Cognitive Science
Societ
- …