6 research outputs found
Expectations or Guarantees? I Want It All! A crossroad between games and MDPs
When reasoning about the strategic capabilities of an agent, it is important
to consider the nature of its adversaries. In the particular context of
controller synthesis for quantitative specifications, the usual problem is to
devise a strategy for a reactive system which yields some desired performance,
taking into account the possible impact of the environment of the system. There
are at least two ways to look at this environment. In the classical analysis of
two-player quantitative games, the environment is purely antagonistic and the
problem is to provide strict performance guarantees. In Markov decision
processes, the environment is seen as purely stochastic: the aim is then to
optimize the expected payoff, with no guarantee on individual outcomes.
In this expository work, we report on recent results introducing the beyond
worst-case synthesis problem, which is to construct strategies that guarantee
some quantitative requirement in the worst-case while providing an higher
expected value against a particular stochastic model of the environment given
as input. This problem is relevant to produce system controllers that provide
nice expected performance in the everyday situation while ensuring a strict
(but relaxed) performance threshold even in the event of very bad (while
unlikely) circumstances. It has been studied for both the mean-payoff and the
shortest path quantitative measures.Comment: In Proceedings SR 2014, arXiv:1404.041
Meet Your Expectations With Guarantees: Beyond Worst-Case Synthesis in Quantitative Games
Classical analysis of two-player quantitative games involves an adversary (modeling the environment of the system) which is purely antagonistic and asks for strict guarantees while Markov decision processes model systems facing a purely randomized environment: the aim is then to optimize the expected payoff, with no guarantee on individual outcomes. We introduce the beyond worst-case synthesis problem, which is to construct strategies that guarantee some quantitative requirement in the worst-case while providing an higher expected value against a particular stochastic model of the environment given as input. We consider both the mean-payoff value problem and the shortest path problem. In both cases, we show how to decide the existence of finite-memory strategies satisfying the problem and how to synthesize one if one exists. We establish algorithms and we study complexity bounds and memory requirements
Percentile Queries in Multi-Dimensional Markov Decision Processes
Markov decision processes (MDPs) with multi-dimensional weights are useful to
analyze systems with multiple objectives that may be conflicting and require
the analysis of trade-offs. We study the complexity of percentile queries in
such MDPs and give algorithms to synthesize strategies that enforce such
constraints. Given a multi-dimensional weighted MDP and a quantitative payoff
function , thresholds (one per dimension), and probability thresholds
, we show how to compute a single strategy to enforce that for all
dimensions , the probability of outcomes satisfying is at least . We consider classical quantitative payoffs from
the literature (sup, inf, lim sup, lim inf, mean-payoff, truncated sum,
discounted sum). Our work extends to the quantitative case the multi-objective
model checking problem studied by Etessami et al. in unweighted MDPs.Comment: Extended version of CAV 2015 pape