A generalized characterization of algorithmic probability

An a priori semimeasure (also known as "algorithmic probability" or "the Solomonoff prior" in the context of inductive inference) is defined as the transformation, by a given universal monotone Turing machine, of the uniform measure on the infinite strings. It is shown in this paper that the class of a priori semimeasures can equivalently be defined as the class of transformations, by all compatible universal monotone Turing machines, of any continuous computable measure in place of the uniform measure. Some consideration is given to possible implications for the prevalent association of algorithmic probability with certain foundational statistical principles

Random strings and tt-degrees of Turing complete C.E. sets

We investigate the truth-table degrees of (co-)c.e.\ sets, in particular, sets of random strings. It is known that the set of random strings with respect to any universal prefix-free machine is Turing complete, but that truth-table completeness depends on the choice of universal machine. We show that for such sets of random strings, any finite set of their truth-table degrees do not meet to the degree~0, even within the c.e. truth-table degrees, but when taking the meet over all such truth-table degrees, the infinite meet is indeed~0. The latter result proves a conjecture of Allender, Friedman and Gasarch. We also show that there are two Turing complete c.e. sets whose truth-table degrees form a minimal pair.Comment: 25 page

Reductions to the set of random strings:the resource-bounded case

This paper is motivated by a conjecture \cite{cie,adfht} that \BPP can be characterized in terms of polynomial-time nonadaptive reductions to the set of Kolmogorov-random strings. In this paper we show that an approach laid out in \cite{adfht} to settle this conjecture cannot succeed without significant alteration, but that it does bear fruit if we consider time-bounded Kolmogorov complexity instead. We show that if a set $A$ is reducible in polynomial time to the set of time-$t$-bounded Kolmogorov-random strings (for all large enough time bounds $t$), then $A$ is in \Ppoly, and that if in addition such a reduction exists for any universal Turing machine one uses in the definition of Kolmogorov complexity, then $A$ is in \PSPACE

Reductions to the set of random strings: The resource-bounded case

This paper is motivated by a conjecture that BPP can be characterized in terms of polynomial-time nonadaptive reductions to the set of Kolmogorov-random strings. In this paper we show that an approach laid out in [Allender et al] to settle this conjecture cannot succeed without significant alteration, but that it does bear fruit if we consider time-bounded Kolmogorov complexity instead. We show that if a set A is reducible in polynomial time to the set of time-t-bounded Kolmogorov random strings (for all large enough time bounds t), then A is in P/poly, and that if in addition such a reduction exists for any universal Turing machine one uses in the definition of Kolmogorov complexity, then A is in PSPACE.Comment: Conference version in MFCS 201

Universal Prediction

In this thesis I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also relates to modern-day speculation about a fully automatized data-driven science. I investigate, in particular, a proposed definition of a universal prediction method that goes back to Solomonoff (1964) and Levin (1970). This definition marks the birth of the theory of Kolmogorov complexity, and has a direct line to the information-theoretic approach in modern machine learning. Solomonoff's work was inspired by Carnap's program of inductive logic, and the more precise definition due to Levin can be seen as an explicit attempt to escape the diagonal argument that Putnam (1963) famously launched against the feasibility of Carnap's program. The Solomonoff-Levin definition essentially aims at a mixture of all possible prediction algorithms. An alternative interpretation is that the definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations. The conclusions of my investigation are negative. I show that the Solomonoff-Levin definition fails to unite two necessary conditions to count as a universal prediction method, as turns out be entailed by Putnam's original argument after all; and I argue that this indeed shows that no definition can. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is already problematic itself

Universal Prediction

In this dissertation I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn what there is to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also touches on modern-day speculation about a fully automatized data-driven science. I investigate, in particular, a specific mathematical definition of a universal prediction method, that goes back to the early days of artificial intelligence and that has a direct line to modern developments in machine learning. This definition essentially aims to combine all possible prediction algorithms. An alternative interpretation is that this definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations. The conclusions of my investigation are negative. I show that the proposed definition cannot be interpreted as a universal prediction method, as turns out to be exposed by a mathematical argument that it was actually intended to overcome. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is problematic itself

Randomness and Computability

This thesis establishes significant new results in the area of algorithmic randomness. These results elucidate the deep relationship between randomness and computability. A number of results focus on randomness for finite strings. Levin introduced two functions which measure the randomness of finite strings. One function is derived from a universal monotone machine and the other function is derived from an optimal computably enumerable semimeasure. Gacs proved that infinitely often, the gap between these two functions exceeds the inverse Ackermann function (applied to string length). This thesis improves this result to show that infinitely often the difference between these two functions exceeds the double logarithm. Another separation result is proved for two different kinds of process machine. Information about the randomness of finite strings can be used as a computational resource. This information is contained in the overgraph. Muchnik and Positselsky asked whether there exists an optimal monotone machine whose overgraph is not truth-table complete. This question is answered in the negative. Related results are also established. This thesis makes advances in the theory of randomness for infinite binary sequences. A variant of process machines is used to characterise computable randomness, Schnorr randomness and weak randomness. This result is extended to give characterisations of these types of randomness using truthtable reducibility. The computable Lipschitz reducibility measures both the relative randomness and the relative computational power of real numbers. It is proved that the computable Lipschitz degrees of computably enumerable sets are not dense. Infinite binary sequences can be regarded as elements of Cantor space. Most research in randomness for Cantor space has been conducted using the uniform measure. However, the study of non-computable measures has led to interesting results. This thesis shows that the two approaches that have been used to define randomness on Cantor space for non-computable measures: that of Reimann and Slaman, along with the uniform test approach first introduced by Levin and also used by Gacs, Hoyrup and Rojas, are equivalent. Levin established the existence of probability measures for which all infinite sequences are random. These measures are termed neutral measures. It is shown that every PA degree computes a neutral measure. Work of Miller is used to show that the set of atoms of a neutral measure is a countable Scott set and in fact any countable Scott set is the set of atoms of some neutral measure. Neutral measures are used to prove new results in computability theory. For example, it is shown that the low computable enumerable sets are precisely the computably enumerable sets bounded by PA degrees strictly below the halting problem. This thesis applies ideas developed in the study of randomness to computability theory by examining indifferent sets for comeager classes in Cantor space. A number of results are proved. For example, it is shown that there exist 1-generic sets that can compute their own indifferent sets