337,105 research outputs found
Learning Action Models: Qualitative Approach
In dynamic epistemic logic, actions are described using action models. In
this paper we introduce a framework for studying learnability of action models
from observations. We present first results concerning propositional action
models. First we check two basic learnability criteria: finite identifiability
(conclusively inferring the appropriate action model in finite time) and
identifiability in the limit (inconclusive convergence to the right action
model). We show that deterministic actions are finitely identifiable, while
non-deterministic actions require more learning power-they are identifiable in
the limit. We then move on to a particular learning method, which proceeds via
restriction of a space of events within a learning-specific action model. This
way of learning closely resembles the well-known update method from dynamic
epistemic logic. We introduce several different learning methods suited for
finite identifiability of particular types of deterministic actions.Comment: 18 pages, accepted for LORI-V: The Fifth International Conference on
Logic, Rationality and Interaction, October 28-31, 2015, National Taiwan
University, Taipei, Taiwa
A Class of Logistic Functions for Approximating State-Inclusive Koopman Operators
An outstanding challenge in nonlinear systems theory is identification or
learning of a given nonlinear system's Koopman operator directly from data or
models. Advances in extended dynamic mode decomposition approaches and machine
learning methods have enabled data-driven discovery of Koopman operators, for
both continuous and discrete-time systems. Since Koopman operators are often
infinite-dimensional, they are approximated in practice using
finite-dimensional systems. The fidelity and convergence of a given
finite-dimensional Koopman approximation is a subject of ongoing research. In
this paper we introduce a class of Koopman observable functions that confer an
approximate closure property on their corresponding finite-dimensional
approximations of the Koopman operator. We derive error bounds for the fidelity
of this class of observable functions, as well as identify two key learning
parameters which can be used to tune performance. We illustrate our approach on
two classical nonlinear system models: the Van Der Pol oscillator and the
bistable toggle switch.Comment: 8 page
Learning Generative ConvNets via Multi-grid Modeling and Sampling
This paper proposes a multi-grid method for learning energy-based generative
ConvNet models of images. For each grid, we learn an energy-based probabilistic
model where the energy function is defined by a bottom-up convolutional neural
network (ConvNet or CNN). Learning such a model requires generating synthesized
examples from the model. Within each iteration of our learning algorithm, for
each observed training image, we generate synthesized images at multiple grids
by initializing the finite-step MCMC sampling from a minimal 1 x 1 version of
the training image. The synthesized image at each subsequent grid is obtained
by a finite-step MCMC initialized from the synthesized image generated at the
previous coarser grid. After obtaining the synthesized examples, the parameters
of the models at multiple grids are updated separately and simultaneously based
on the differences between synthesized and observed examples. We show that this
multi-grid method can learn realistic energy-based generative ConvNet models,
and it outperforms the original contrastive divergence (CD) and persistent CD.Comment: CVPR 201
The Sample-Complexity of General Reinforcement Learning
We present a new algorithm for general reinforcement learning where the true
environment is known to belong to a finite class of N arbitrary models. The
algorithm is shown to be near-optimal for all but O(N log^2 N) time-steps with
high probability. Infinite classes are also considered where we show that
compactness is a key criterion for determining the existence of uniform
sample-complexity bounds. A matching lower bound is given for the finite case.Comment: 16 page
Constant Step Size Stochastic Gradient Descent for Probabilistic Modeling
Stochastic gradient methods enable learning probabilistic models from large
amounts of data. While large step-sizes (learning rates) have shown to be best
for least-squares (e.g., Gaussian noise) once combined with parameter
averaging, these are not leading to convergent algorithms in general. In this
paper, we consider generalized linear models, that is, conditional models based
on exponential families. We propose averaging moment parameters instead of
natural parameters for constant-step-size stochastic gradient descent. For
finite-dimensional models, we show that this can sometimes (and surprisingly)
lead to better predictions than the best linear model. For infinite-dimensional
models, we show that it always converges to optimal predictions, while
averaging natural parameters never does. We illustrate our findings with
simulations on synthetic data and classical benchmarks with many observations.Comment: Published in Proc. UAI 2018, was accepted as oral presentation Camera
read
Learning Markov Decision Processes for Model Checking
Constructing an accurate system model for formal model verification can be
both resource demanding and time-consuming. To alleviate this shortcoming,
algorithms have been proposed for automatically learning system models based on
observed system behaviors. In this paper we extend the algorithm on learning
probabilistic automata to reactive systems, where the observed system behavior
is in the form of alternating sequences of inputs and outputs. We propose an
algorithm for automatically learning a deterministic labeled Markov decision
process model from the observed behavior of a reactive system. The proposed
learning algorithm is adapted from algorithms for learning deterministic
probabilistic finite automata, and extended to include both probabilistic and
nondeterministic transitions. The algorithm is empirically analyzed and
evaluated by learning system models of slot machines. The evaluation is
performed by analyzing the probabilistic linear temporal logic properties of
the system as well as by analyzing the schedulers, in particular the optimal
schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345
- …