2 research outputs found
Symbolic Controller Synthesis for B\"uchi Specifications on Stochastic Systems
We consider the policy synthesis problem for continuous-state controlled
Markov processes evolving in discrete time, when the specification is given as
a B\"uchi condition (visit a set of states infinitely often). We decompose
computation of the maximal probability of satisfying the B\"uchi condition into
two steps. The first step is to compute the maximal qualitative winning set,
from where the B\"uchi condition can be enforced with probability one. The
second step is to find the maximal probability of reaching the already computed
qualitative winning set. In contrast with finite-state models, we show that
such a computation only gives a lower bound on the maximal probability where
the gap can be non-zero.
In this paper we focus on approximating the qualitative winning set, while
pointing out that the existing approaches for unbounded reachability
computation can solve the second step. We provide an abstraction-based
technique to approximate the qualitative winning set by simultaneously using an
over- and under-approximation of the probabilistic transition relation. Since
we are interested in qualitative properties, the abstraction is
non-probabilistic; instead, the probabilistic transitions are assumed to be
under the control of a (fair) adversary. Thus, we reduce the original policy
synthesis problem to a B\"uchi game under a fairness assumption and
characterize upper and lower bounds on winning sets as nested fixed point
expressions in the -calculus. This characterization immediately provides a
symbolic algorithm scheme. Further, a winning strategy computed on the abstract
game can be refined to a policy on the controlled Markov process.
We describe a concrete abstraction procedure and demonstrate our algorithm on
two case studies