4,257 research outputs found
Convergence in Models with Bounded Expected Relative Hazard Rates
We provide a general framework to study stochastic sequences related to
individual learning in economics, learning automata in computer sciences,
social learning in marketing, and other applications. More precisely, we study
the asymptotic properties of a class of stochastic sequences that take values
in and satisfy a property called "bounded expected relative hazard
rates." Sequences that satisfy this property and feature "small step-size" or
"shrinking step-size" converge to 1 with high probability or almost surely,
respectively. These convergence results yield conditions for the learning
models in B\"orgers, Morales, and Sarin (2004), Erev and Roth (1998), and
Schlag (1998) to choose expected payoff maximizing actions with probability one
in the long run.Comment: After revision. Accepted for publication by Journal of Economic
Theor
Learning probability distributions generated by finite-state machines
We review methods for inference of probability distributions generated by probabilistic automata and related models for sequence generation. We focus on methods that can be proved to learn in the inference
in the limit and PAC formal models. The methods we review are state merging and state splitting methods for probabilistic deterministic automata and the recently developed spectral method for nondeterministic probabilistic automata. In both cases, we derive them from a high-level algorithm described in terms of the Hankel matrix of the distribution to be learned, given as an oracle, and then describe how to adapt that algorithm to account for the error introduced by a finite sample.Peer ReviewedPostprint (author's final draft
The Complexity of POMDPs with Long-run Average Objectives
We study the problem of approximation of optimal values in
partially-observable Markov decision processes (POMDPs) with long-run average
objectives. POMDPs are a standard model for dynamic systems with probabilistic
and nondeterministic behavior in uncertain environments. In long-run average
objectives rewards are associated with every transition of the POMDP and the
payoff is the long-run average of the rewards along the executions of the
POMDP. We establish strategy complexity and computational complexity results.
Our main result shows that finite-memory strategies suffice for approximation
of optimal values, and the related decision problem is recursively enumerable
complete
Coding-theorem Like Behaviour and Emergence of the Universal Distribution from Resource-bounded Algorithmic Probability
Previously referred to as `miraculous' in the scientific literature because
of its powerful properties and its wide application as optimal solution to the
problem of induction/inference, (approximations to) Algorithmic Probability
(AP) and the associated Universal Distribution are (or should be) of the
greatest importance in science. Here we investigate the emergence, the rates of
emergence and convergence, and the Coding-theorem like behaviour of AP in
Turing-subuniversal models of computation. We investigate empirical
distributions of computing models in the Chomsky hierarchy. We introduce
measures of algorithmic probability and algorithmic complexity based upon
resource-bounded computation, in contrast to previously thoroughly investigated
distributions produced from the output distribution of Turing machines. This
approach allows for numerical approximations to algorithmic
(Kolmogorov-Chaitin) complexity-based estimations at each of the levels of a
computational hierarchy. We demonstrate that all these estimations are
correlated in rank and that they converge both in rank and values as a function
of computational power, despite fundamental differences between computational
models. In the context of natural processes that operate below the Turing
universal level because of finite resources and physical degradation, the
investigation of natural biases stemming from algorithmic rules may shed light
on the distribution of outcomes. We show that up to 60\% of the
simplicity/complexity bias in distributions produced even by the weakest of the
computational models can be accounted for by Algorithmic Probability in its
approximation to the Universal Distribution.Comment: 27 pages main text, 39 pages including supplement. Online complexity
calculator: http://complexitycalculator.com
Learning Residual Finite-State Automata Using Observation Tables
We define a two-step learner for RFSAs based on an observation table by using
an algorithm for minimal DFAs to build a table for the reversal of the language
in question and showing that we can derive the minimal RFSA from it after some
simple modifications. We compare the algorithm to two other table-based ones of
which one (by Bollig et al. 2009) infers a RFSA directly, and the other is
another two-step learner proposed by the author. We focus on the criterion of
query complexity.Comment: In Proceedings DCFS 2010, arXiv:1008.127
- …