26 research outputs found
Stochastic Games with Disjunctions of Multiple Objectives (Technical Report)
Stochastic games combine controllable and adversarial non-determinism with
stochastic behavior and are a common tool in control, verification and
synthesis of reactive systems facing uncertainty. Multi-objective stochastic
games are natural in situations where several - possibly conflicting -
performance criteria like time and energy consumption are relevant. Such
conjunctive combinations are the most studied multi-objective setting in the
literature. In this paper, we consider the dual disjunctive problem. More
concretely, we study turn-based stochastic two-player games on graphs where the
winning condition is to guarantee at least one reachability or safety objective
from a given set of alternatives. We present a fine-grained overview of
strategy and computational complexity of such \emph{disjunctive queries} (DQs)
and provide new lower and upper bounds for several variants of the problem,
significantly extending previous works. We also propose a novel value
iteration-style algorithm for approximating the set of Pareto optimal
thresholds for a given DQ.Comment: Technical report including appendix with detailed proofs, 29 page
Automata Tutor v3
Computer science class enrollments have rapidly risen in the past decade.
With current class sizes, standard approaches to grading and providing
personalized feedback are no longer possible and new techniques become both
feasible and necessary. In this paper, we present the third version of Automata
Tutor, a tool for helping teachers and students in large courses on automata
and formal languages. The second version of Automata Tutor supported automatic
grading and feedback for finite-automata constructions and has already been
used by thousands of users in dozens of countries. This new version of Automata
Tutor supports automated grading and feedback generation for a greatly extended
variety of new problems, including problems that ask students to create regular
expressions, context-free grammars, pushdown automata and Turing machines
corresponding to a given description, and problems about converting between
equivalent models - e.g., from regular expressions to nondeterministic finite
automata. Moreover, for several problems, this new version also enables
teachers and students to automatically generate new problem instances. We also
present the results of a survey run on a class of 950 students, which shows
very positive results about the usability and usefulness of the tool
Anytime Guarantees for Reachability in Uncountable Markov Decision Processes
We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a sequence of approximations converging to the true value in the limit, our aim is to obtain an algorithm with guarantees on the precision of the approximation.
As this problem is undecidable in general, assumptions on the MDP are necessary. Our main contribution is to identify sufficient assumptions that are as weak as possible, thus approaching the "boundary" of which systems can be correctly and reliably analyzed. To this end, we also argue why each of our assumptions is necessary for algorithms based on processing finitely many observations.
We present two solution variants. The first one provides converging lower bounds under weaker assumptions than typical ones from previous works concerned with guarantees. The second one then utilizes stronger assumptions to additionally provide converging upper bounds. Altogether, we obtain an anytime algorithm, i.e. yielding a sequence of approximants with known and iteratively improving precision, converging to the true value in the limit. Besides, due to the generality of our assumptions, our algorithms are very general templates, readily allowing for various heuristics from literature in contrast to, e.g., a specific discretization algorithm. Our theoretical contribution thus paves the way for future practical improvements without sacrificing correctness guarantees
Enforcing ?-Regular Properties in Markov Chains by Restarting
Restarts are used in many computer systems to improve performance. Examples include reloading a webpage, reissuing a request, or restarting a randomized search. The design of restart strategies has been extensively studied by the performance evaluation community. In this paper, we address the problem of designing universal restart strategies, valid for arbitrary finite-state Markov chains, that enforce a given ?-regular property while not knowing the chain. A strategy enforces a property ? if, with probability 1, the number of restarts is finite, and the run of the Markov chain after the last restart satisfies ?. We design a simple "cautious" strategy that solves the problem, and a more sophisticated "bold" strategy with an almost optimal number of restarts
An Anytime Algorithm for Reachability on Uncountable MDP
We provide an algorithm for reachability on Markov decision processes with
uncountable state and action spaces, which, under mild assumptions,
approximates the optimal value to any desired precision. It is the first such
anytime algorithm, meaning that at any point in time it can return the current
approximation with its precision. Moreover, it simultaneously is the first
algorithm able to utilize \emph{learning} approaches without sacrificing
guarantees and it further allows for combination with existing heuristics
Index appearance record with preorders
Transforming ω-automata into parity automata is traditionally done using appearance records. We present an efficient variant of this idea, tailored to Rabin automata, and several optimizations applicable to all appearance records. We compare the methods experimentally and show that our method produces significantly smaller automata than previous approaches
Comparison of Algorithms for Simple Stochastic Games (Full Version)
Simple stochastic games are turn-based 2.5-player zero-sum graph games with a
reachability objective. The problem is to compute the winning probability as
well as the optimal strategies of both players. In this paper, we compare the
three known classes of algorithms -- value iteration, strategy iteration and
quadratic programming -- both theoretically and practically. Further, we
suggest several improvements for all algorithms, including the first approach
based on quadratic programming that avoids transforming the stochastic game to
a stopping one. Our extensive experiments show that these improvements can lead
to significant speed-ups. We implemented all algorithms in PRISM-games 3.0,
thereby providing the first implementation of quadratic programming for solving
simple stochastic games