2,270 research outputs found
The Stochastic Shortest Path Problem : A polyhedral combinatorics perspective
In this paper, we give a new framework for the stochastic shortest path
problem in finite state and action spaces. Our framework generalizes both the
frameworks proposed by Bertsekas and Tsitsikli and by Bertsekas and Yu. We
prove that the problem is well-defined and (weakly) polynomial when (i) there
is a way to reach the target state from any initial state and (ii) there is no
transition cycle of negative costs (a generalization of negative cost cycles).
These assumptions generalize the standard assumptions for the deterministic
shortest path problem and our framework encapsulates the latter problem (in
contrast with prior works). In this new setting, we can show that (a) one can
restrict to deterministic and stationary policies, (b) the problem is still
(weakly) polynomial through linear programming, (c) Value Iteration and Policy
Iteration converge, and (d) we can extend Dijkstra's algorithm
Efficient computation of exact solutions for quantitative model checking
Quantitative model checkers for Markov Decision Processes typically use
finite-precision arithmetic. If all the coefficients in the process are
rational numbers, then the model checking results are rational, and so they can
be computed exactly. However, exact techniques are generally too expensive or
limited in scalability. In this paper we propose a method for obtaining exact
results starting from an approximated solution in finite-precision arithmetic.
The input of the method is a description of a scheduler, which can be obtained
by a model checker using finite precision. Given a scheduler, we show how to
obtain a corresponding basis in a linear-programming problem, in such a way
that the basis is optimal whenever the scheduler attains the worst-case
probability. This correspondence is already known for discounted MDPs, we show
how to apply it in the undiscounted case provided that some preprocessing is
done. Using the correspondence, the linear-programming problem can be solved in
exact arithmetic starting from the basis obtained. As a consequence, the method
finds the worst-case probability even if the scheduler provided by the model
checker was not optimal. In our experiments, the calculation of exact solutions
from a candidate scheduler is significantly faster than the calculation using
the simplex method under exact arithmetic starting from a default basis.Comment: In Proceedings QAPL 2012, arXiv:1207.055
Three Puzzles on Mathematics, Computation, and Games
In this lecture I will talk about three mathematical puzzles involving
mathematics and computation that have preoccupied me over the years. The first
puzzle is to understand the amazing success of the simplex algorithm for linear
programming. The second puzzle is about errors made when votes are counted
during elections. The third puzzle is: are quantum computers possible?Comment: ICM 2018 plenary lecture, Rio de Janeiro, 36 pages, 7 Figure
- …