13,207 research outputs found
On pattern classification algorithms - Introduction and survey
Pattern recognition algorithms, and mathematical techniques of estimation, decision making, and optimization theor
Multiscale Markov Decision Problems: Compression, Solution, and Transfer Learning
Many problems in sequential decision making and stochastic control often have
natural multiscale structure: sub-tasks are assembled together to accomplish
complex goals. Systematically inferring and leveraging hierarchical structure,
particularly beyond a single level of abstraction, has remained a longstanding
challenge. We describe a fast multiscale procedure for repeatedly compressing,
or homogenizing, Markov decision processes (MDPs), wherein a hierarchy of
sub-problems at different scales is automatically determined. Coarsened MDPs
are themselves independent, deterministic MDPs, and may be solved using
existing algorithms. The multiscale representation delivered by this procedure
decouples sub-tasks from each other and can lead to substantial improvements in
convergence rates both locally within sub-problems and globally across
sub-problems, yielding significant computational savings. A second fundamental
aspect of this work is that these multiscale decompositions yield new transfer
opportunities across different problems, where solutions of sub-tasks at
different levels of the hierarchy may be amenable to transfer to new problems.
Localized transfer of policies and potential operators at arbitrary scales is
emphasized. Finally, we demonstrate compression and transfer in a collection of
illustrative domains, including examples involving discrete and continuous
statespaces.Comment: 86 pages, 15 figure
Different Approaches on Stochastic Reachability as an Optimal Stopping Problem
Reachability analysis is the core of model checking of time systems. For
stochastic hybrid systems, this safety verification method is very little supported mainly
because of complexity and difficulty of the associated mathematical problems. In this
paper, we develop two main directions of studying stochastic reachability as an optimal
stopping problem. The first approach studies the hypotheses for the dynamic programming
corresponding with the optimal stopping problem for stochastic hybrid systems.
In the second approach, we investigate the reachability problem considering approximations
of stochastic hybrid systems. The main difficulty arises when we have to prove the
convergence of the value functions of the approximating processes to the value function
of the initial process. An original proof is provided
New insights on stochastic reachability
In this paper, we give new characterizations of the stochastic reachability problem for stochastic hybrid systems in the language of different theories that can be employed in studying stochastic processes (Markov processes, potential theory, optimal control). These characterizations are further used to obtain the probabilities involved in the context of stochastic reachability as viscosity solutions of some variational inequalities
- ā¦