182 research outputs found
Dimension Extractors and Optimal Decompression
A *dimension extractor* is an algorithm designed to increase the effective
dimension -- i.e., the amount of computational randomness -- of an infinite
binary sequence, in order to turn a "partially random" sequence into a "more
random" sequence. Extractors are exhibited for various effective dimensions,
including constructive, computable, space-bounded, time-bounded, and
finite-state dimension. Using similar techniques, the Kucera-Gacs theorem is
examined from the perspective of decompression, by showing that every infinite
sequence S is Turing reducible to a Martin-Loef random sequence R such that the
asymptotic number of bits of R needed to compute n bits of S, divided by n, is
precisely the constructive dimension of S, which is shown to be the optimal
ratio of query bits to computed bits achievable with Turing reductions. The
extractors and decompressors that are developed lead directly to new
characterizations of some effective dimensions in terms of optimal
decompression by Turing reductions.Comment: This report was combined with a different conference paper "Every
Sequence is Decompressible from a Random One" (cs.IT/0511074, at
http://dx.doi.org/10.1007/11780342_17), and both titles were changed, with
the conference paper incorporated as section 5 of this new combined paper.
The combined paper was accepted to the journal Theory of Computing Systems,
as part of a special issue of invited papers from the second conference on
Computability in Europe, 200
Extending the Reach of the Point-To-Set Principle
The point-to-set principle of J. Lutz and N. Lutz (2018) has recently enabled
the theory of computing to be used to answer open questions about fractal
geometry in Euclidean spaces . These are classical questions,
meaning that their statements do not involve computation or related aspects of
logic.
In this paper we extend the reach of the point-to-set principle from
Euclidean spaces to arbitrary separable metric spaces . We first extend two
fractal dimensions--computability-theoretic versions of classical Hausdorff and
packing dimensions that assign dimensions and to
individual points --to arbitrary separable metric spaces and to
arbitrary gauge families. Our first two main results then extend the
point-to-set principle to arbitrary separable metric spaces and to a large
class of gauge families.
We demonstrate the power of our extended point-to-set principle by using it
to prove new theorems about classical fractal dimensions in hyperspaces. (For a
concrete computational example, the stages used to
construct a self-similar fractal in the plane are elements of the
hyperspace of the plane, and they converge to in the hyperspace.) Our third
main result, proven via our extended point-to-set principle, states that, under
a wide variety of gauge families, the classical packing dimension agrees with
the classical upper Minkowski dimension on all hyperspaces of compact sets. We
use this theorem to give, for all sets that are analytic, i.e.,
, a tight bound on the packing dimension of the hyperspace
of in terms of the packing dimension of itself
The complexity of parameters for probabilistic and quantum computation
In this dissertation we study some effects of allowing computational models that use parameters whose own computational complexity has a strong effect on the computational complexity of the languages computable from the model. We show that in the probabilistic and quantum models there are parameter sets that allow one to obtain noncomputable outcomes;In Chapter 3 we define BP[beta]P the BPP class based on a coin with bias [beta]. We then show that if [beta] is BPP-computable then it is the case that BP[beta]P = BPP. We also show that each language L in P/CLog is in BP[beta]P for some [beta]. Hence there are some [beta] from which we can compute noncomputable languages. We also examine the robustness of the class BPP with respect to small variations from fairness in the coin;In Chapter 4 we consider measures that are based on polynomial-time computable sequences of biased coins in which the biases are bounded away from both zero and one (strongly positive P-sequences). We show that such a sequence [vector][beta] generates a measure [mu][vector][beta] equivalent to the uniform measure in the sense that if C is a class of languages closed under positive, polynomial-time, truth-table reductions with queries of linear length then C has [mu][vector][beta]-measure zero if and only if it has measure zero relative to the uniform measure [mu]. The classes P, NP, BPP, P/Poly, PH, and PSPACE are among those to which this result applies. Thus the measures of these much-studied classes are robust with respect to changes of this type in the underlying probability measure;In Chapter 5 we introduce the quantum computation model and the quantum complexity class BQP. We claim that the computational complexity of the amplitudes is a critical factor in determining the languages computable using the quantum model. Using results from chapter 3 we show that the quantum model can also compute noncomputable languages from some amplitude sets. Finally, we determine a restriction on the amplitude set to limit the model to the range of languages implicit in others\u27 typical meaning of the class BQP
Universal Prediction
In this thesis I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also relates to modern-day speculation about a fully automatized data-driven science.
I investigate, in particular, a proposed definition of a universal prediction method that goes back to Solomonoff (1964) and Levin (1970). This definition marks the birth of the theory of Kolmogorov complexity, and has a direct line to the information-theoretic approach in modern machine learning. Solomonoff's work was inspired by Carnap's program of inductive logic, and the more precise definition due to Levin can be seen as an explicit attempt to escape the diagonal argument that Putnam (1963) famously launched against the feasibility of Carnap's program.
The Solomonoff-Levin definition essentially aims at a mixture of all possible prediction algorithms. An alternative interpretation is that the definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations.
The conclusions of my investigation are negative. I show that the Solomonoff-Levin definition fails to unite two necessary conditions to count as a universal prediction method, as turns out be entailed by Putnam's original argument after all; and I argue that this indeed shows that no definition can. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is already problematic itself
Universal Prediction
In this dissertation I investigate the theoretical possibility of a universal method of prediction. A prediction method is universal if it is always able to learn what there is to learn from data: if it is always able to extrapolate given data about past observations to maximally successful predictions about future observations. The context of this investigation is the broader philosophical question into the possibility of a formal specification of inductive or scientific reasoning, a question that also touches on modern-day speculation about a fully automatized data-driven science.
I investigate, in particular, a specific mathematical definition of a universal prediction method, that goes back to the early days of artificial intelligence and that has a direct line to modern developments in machine learning. This definition essentially aims to combine all possible prediction algorithms. An alternative interpretation is that this definition formalizes the idea that learning from data is equivalent to compressing data. In this guise, the definition is often presented as an implementation and even as a justification of Occam's razor, the principle that we should look for simple explanations.
The conclusions of my investigation are negative. I show that the proposed definition cannot be interpreted as a universal prediction method, as turns out to be exposed by a mathematical argument that it was actually intended to overcome. Moreover, I show that the suggested justification of Occam's razor does not work, and I argue that the relevant notion of simplicity as compressibility is problematic itself
Nonparametric General Reinforcement Learning
Reinforcement learning problems are often phrased in terms of
Markov decision processes (MDPs). In this thesis we go beyond
MDPs and consider reinforcement learning in environments that are
non-Markovian, non-ergodic and only partially observable. Our
focus is not on practical algorithms, but rather on the
fundamental underlying problems: How do we balance exploration
and exploitation? How do we explore optimally? When is an agent
optimal? We follow the nonparametric realizable paradigm: we
assume the data is drawn from an unknown source that belongs to a
known countable class of candidates.
First, we consider the passive (sequence prediction) setting,
learning from data that is not independent and identically
distributed. We collect results from artificial intelligence,
algorithmic information theory, and game theory and put them in a
reinforcement learning context: they demonstrate how an agent can
learn the value of its own policy.
Next, we establish negative results on Bayesian reinforcement
learning agents, in particular AIXI. We show that unlucky or
adversarial choices of the prior cause the agent to misbehave
drastically. Therefore Legg-Hutter intelligence and balanced
Pareto optimality, which depend crucially on the choice of the
prior, are entirely subjective. Moreover, in the class of all
computable environments every policy is Pareto optimal. This
undermines all existing optimality properties for AIXI.
However, there are Bayesian approaches to general reinforcement
learning that satisfy objective optimality guarantees: We prove
that Thompson sampling
is asymptotically optimal in stochastic environments in the sense
that its value converges to the value of the optimal policy. We
connect asymptotic optimality to regret
given a recoverability assumption on the environment that allows
the agent to recover from mistakes. Hence Thompson sampling
achieves sublinear regret in these environments.
AIXI is known to be incomputable. We quantify this using the
arithmetical hierarchy, and establish upper and corresponding
lower bounds for incomputability. Further, we show that AIXI is
not limit computable, thus cannot be approximated using finite
computation. However there are limit computable ε-optimal
approximations to AIXI. We also derive computability bounds for
knowledge-seeking agents, and give a limit computable weakly
asymptotically optimal reinforcement learning agent.
Finally, our results culminate in a formal solution to the grain
of truth problem: A Bayesian agent acting in a multi-agent
environment learns to predict the other agents' policies if its
prior assigns positive probability to them (the prior contains a
grain of truth). We construct a large but limit computable class
containing a grain of truth
and show that agents based on Thompson sampling over this class
converge to play ε-Nash equilibria in arbitrary unknown
computable multi-agent environments
Nonparametric General Reinforcement Learning
Reinforcement learning problems are often phrased in terms of
Markov decision processes (MDPs). In this thesis we go beyond
MDPs and consider reinforcement learning in environments that are
non-Markovian, non-ergodic and only partially observable. Our
focus is not on practical algorithms, but rather on the
fundamental underlying problems: How do we balance exploration
and exploitation? How do we explore optimally? When is an agent
optimal? We follow the nonparametric realizable paradigm: we
assume the data is drawn from an unknown source that belongs to a
known countable class of candidates.
First, we consider the passive (sequence prediction) setting,
learning from data that is not independent and identically
distributed. We collect results from artificial intelligence,
algorithmic information theory, and game theory and put them in a
reinforcement learning context: they demonstrate how an agent can
learn the value of its own policy.
Next, we establish negative results on Bayesian reinforcement
learning agents, in particular AIXI. We show that unlucky or
adversarial choices of the prior cause the agent to misbehave
drastically. Therefore Legg-Hutter intelligence and balanced
Pareto optimality, which depend crucially on the choice of the
prior, are entirely subjective. Moreover, in the class of all
computable environments every policy is Pareto optimal. This
undermines all existing optimality properties for AIXI.
However, there are Bayesian approaches to general reinforcement
learning that satisfy objective optimality guarantees: We prove
that Thompson sampling
is asymptotically optimal in stochastic environments in the sense
that its value converges to the value of the optimal policy. We
connect asymptotic optimality to regret
given a recoverability assumption on the environment that allows
the agent to recover from mistakes. Hence Thompson sampling
achieves sublinear regret in these environments.
AIXI is known to be incomputable. We quantify this using the
arithmetical hierarchy, and establish upper and corresponding
lower bounds for incomputability. Further, we show that AIXI is
not limit computable, thus cannot be approximated using finite
computation. However there are limit computable ε-optimal
approximations to AIXI. We also derive computability bounds for
knowledge-seeking agents, and give a limit computable weakly
asymptotically optimal reinforcement learning agent.
Finally, our results culminate in a formal solution to the grain
of truth problem: A Bayesian agent acting in a multi-agent
environment learns to predict the other agents' policies if its
prior assigns positive probability to them (the prior contains a
grain of truth). We construct a large but limit computable class
containing a grain of truth
and show that agents based on Thompson sampling over this class
converge to play ε-Nash equilibria in arbitrary unknown
computable multi-agent environments
- …