Automated verification of concurrent stochastic games

We present automatic verifcation techniques for concurrent stochastic multi-player games (CSGs) with rewards. To express properties of such models, we adapt the temporal logic rPATL (probabilistic alternating-time temporal logic with rewards), originally introduced for the simpler model of turn-based games, which enables quantitative reasoning about the ability of coalitions of players to achieve goals related to the probability of an event or reward measures. We propose and implement a modelling approach and model checking algorithms for property verifcation and strategy synthesis of CSGs, as an extension of PRISMgames. We evaluate the performance, scalability and applicability of our techniques on case studies from domains such as security, networks and finance, showing that we can analyse systems with probabilistic, cooperative and competitive behaviour between concurrent components, including many scenarios that cannot be analysed with turn-based models

Approximating values of generalized-reachability stochastic games

Simple stochastic games are turn-based 2½-player games with a reachability objective. The basic question asks whether one player can ensure reaching a given target with at least a given probability. A natural extension is games with a conjunction of such conditions as objective. Despite a plethora of recent results on the analysis of systems with multiple objectives, the decidability of this basic problem remains open. In this paper, we present an algorithm approximating the Pareto frontier of the achievable values to a given precision. Moreover, it is an anytime algorithm, meaning it can be stopped at any time returning the current approximation and its error bound

Comparison of Algorithms for Simple Stochastic Games (Full Version)

Simple stochastic games are turn-based 2.5-player zero-sum graph games with a reachability objective. The problem is to compute the winning probability as well as the optimal strategies of both players. In this paper, we compare the three known classes of algorithms -- value iteration, strategy iteration and quadratic programming -- both theoretically and practically. Further, we suggest several improvements for all algorithms, including the first approach based on quadratic programming that avoids transforming the stochastic game to a stopping one. Our extensive experiments show that these improvements can lead to significant speed-ups. We implemented all algorithms in PRISM-games 3.0, thereby providing the first implementation of quadratic programming for solving simple stochastic games

Value Iteration for Simple Stochastic Games: Stopping Criterion and Learning Algorithm

Simple stochastic games can be solved by value iteration (VI), which yields a sequence of under-approximations of the value of the game. This sequence is guaranteed to converge to the value only in the limit. Since no stopping criterion is known, this technique does not provide any guarantees on its results. We provide the first stopping criterion for VI on simple stochastic games. It is achieved by additionally computing a convergent sequence of over-approximations of the value, relying on an analysis of the game graph. Consequently, VI becomes an anytime algorithm returning the approximation of the value and the current error bound. As another consequence, we can provide a simulation-based asynchronous VI algorithm, which yields the same guarantees, but without necessarily exploring the whole game graph.Comment: CAV201

PRISM-games: verification and strategy synthesis for stochastic multi-player games with multiple objectives

PRISM-games is a tool for modelling, verification and strategy synthesis for stochastic multi-player games. These allow models to incorporate both probability, to represent uncertainty, unreliability or randomisation, and game-theoretic aspects, for systems where different entities have opposing objectives. Applications include autonomous transport, security protocols, energy management systems and many more. We provide a detailed overview of the PRISM-games tool, including its modelling and property specification formalisms, and its underlying architecture and implementation. In particular, we discuss some of its key features, which include multi-objective and compositional approaches to verification and strategy synthesis. We also discuss the scalability and efficiency of the tool and give an overview of some of the case studies to which it has been applied

PRISM-games:Verification and Strategy Synthesis for Stochastic Multi-player Games with Multiple Objectives

PRISM-games is a tool for modelling, verification and strategy synthesis for stochastic multi-player games. These allow models to incorporate both probability, to represent uncertainty, unreliability or randomisation, and game-theoretic aspects, for systems where different entities have opposing objectives. Applications include autonomous transport, security protocols, energy management systems and many more. We provide a detailed overview of the PRISM-games tool, including its modelling and property specification formalisms, and its underlying architecture and implementation. In particular, we discuss some of its key features, which include multi-objective and compositional approaches to verification and strategy synthesis. We also discuss the scalability and efficiency of the tool and give an overview of some of the case studies to which it has been applied

Automatic Verification of Concurrent Stochastic Systems

Automated verification techniques for stochastic games allow formal reasoning about systems that feature competitive or collaborative behaviour among rational agents in uncertain or probabilistic settings. Existing tools and techniques focus on turn-based games, where each state of the game is controlled by a single player, and on zero-sum properties, where two players or coalitions have directly opposing objectives. In this paper, we present automated verification techniques for concurrent stochastic games (CSGs), which provide a more natural model of concurrent decision making and interaction. We also consider (social welfare) Nash equilibria, to formally identify scenarios where two players or coalitions with distinct goals can collaborate to optimise their joint performance. We propose an extension of the temporal logic rPATL for specifying quantitative properties in this setting and present corresponding algorithms for verification and strategy synthesis for a variant of stopping games. For finite-horizon properties the computation is exact, while for infinite-horizon it is approximate using value iteration. For zero-sum properties it requires solving matrix games via linear programming, and for equilibria-based properties we find social welfare or social cost Nash equilibria of bimatrix games via the method of labelled polytopes through an SMT encoding. We implement this approach in PRISM-games, which required extending the tool's modelling language for CSGs, and apply it to case studies from domains including robotics, computer security and computer networks, explicitly demonstrating the benefits of both CSGs and equilibria-based properties

Automatic verification of concurrent stochastic systems

Automated verification techniques for stochastic games allow formal reasoning about systems that feature competitive or collaborative behaviour among rational agents in uncertain or probabilistic settings. Existing tools and techniques focus on turn-based games, where each state of the game is controlled by a single player, and on zero-sum properties, where two players or coalitions have directly opposing objectives. In this paper, we present automated verification techniques for concurrent stochastic games (CSGs), which provide a more natural model of concurrent decision making and interaction. We also consider (social welfare) Nash equilibria, to formally identify scenarios where two players or coalitions with distinct goals can collaborate to optimise their joint performance. We propose an extension of the temporal logic rPATL for specifying quantitative properties in this setting and present corresponding algorithms for verification and strategy synthesis for a variant of stopping games. For finite-horizon properties the computation is exact, while for infinite-horizon it is approximate using value iteration. For zero-sum properties it requires solving matrix games via linear programming, and for equilibria-based properties we find social welfare or social cost Nash equilibria of bimatrix games via the method of labelled polytopes through an SMT encoding. We implement this approach in PRISM-games, which required extending the tool’s modelling language for CSGs, and apply it to case studies from domains including robotics, computer security and computer networks, explicitly demonstrating the benefits of both CSGs and equilibria-based properties

Synthesis and control of infinite-state systems with partial observability

Complex computer systems play an important role in every part of everyday life and their correctness is often vital to human safety. In light of the recent advances in the area of formal methods and the increasing availability and maturity of tools and techniques, the use of verification techniques to show that a system satisfies a specified property is about to become an integral part of the development process. To minimize the development costs, formal methods must be applied as early as possible, before the entire system is fully developed, or even at the stage when only its specification is available. The goal of synthesis is to automatically construct an implementation guaranteed to fulfill the provided specification, and, if no implementation exists, to report that the given requirements cannot be realized. When synthesizing an individual component within a system and its external environment, the synthesis procedure must take into account the component’s interface and deliver implementations that comply with it. For example, what a component can observe about its environment may be restricted by imprecise sensors or inaccessible communication channels. In addition, sufficiently precise models of a component’s environment are typically infinite-state, for example due to modeling real time or unbounded communication buffers. This thesis presents novel synthesis methods that respect the given interface limitations of the synthesized system components and are applicable to infinite-state models. The studied computational model is that of infinite-state two-player games under incomplete information. The contributions are structured into three parts, corresponding to a classification of such games according to the interface between the synthesized component and its environment. In the first part, we obtain decidability results for a class of game structures where the player corresponding to the synthesized component has a given finite set of possible observations and a finite set of possible actions. A prominent type of systems for which the interface of a component naturally defines a finite set of observations are Lossy Channel Systems. We provide symbolic game solving and strategy synthesis algorithms for lossy channel games under incomplete information with safety and reachability winning conditions. Our second contribution is a counterexample-guided abstraction refinement scheme for solving infinite-state under incomplete information in which the actions available to the component are still finitely many, but no finite set of possible observations is given. This situation is common, for example, in the synthesis of mutex protocols or robot controllers. In this setting, the observations correspond to observation predicates, which are logical formulas, and their computation is an integral part of our synthesis procedure. The resulting game solving method is applicable to games that are out of the scope of other available techniques. Last we study systems in which, in addition to the possibly infinite set of observation predicates, the component can choose between infinitely many possible actions. Timed games under incomplete information are a fundamental class of games for which this is the case. We extend the abstraction-refinement procedure to develop the first systematic method for the synthesis of observation predicates for timed control. Automatically refining the set of candidate observations based on counterexamples demonstrates better potential than brute-force enumeration of observation sets, in particular for systems where fine granularity of the observations is necessary.Komplexe Computer Systeme spielen eine wichtige Rolle in jedem Teil des Alltags und ihre Korrektheit ist oft entscheidend für die menschliche Sicherheit. Angesichts der neuesten Fortschritte auf dem Gebiet der formalen Methoden und die zunehmende Verfügbarkeit und Reife von Tools und Verfahren, wird die Verwendung von Techniken zur Prüfung, dass ein System eine bestimmte Eigenschaft erfüllt, zu einem integralen Bestandteil des Entwicklungsprozesses. Um die Entwicklungskosten zu minimieren, sollen formale Methoden so früh wie möglich angewendet werden, bevor das System vollständig entwickelt ist, oder sogar in der Phase, wenn nur seine Spezifikation zur Verfügung steht. Das Ziel von Synthese ist, automatisch eine Implementierung zu konstruieren, die garantiert die gegebene Spezifikation erfüllt. Falls keine solche Implementierung existiert, soll die Unrealisierbarkeit der Spezifikation ausgewiesen werden. Bei der Synthese einer einzelnen Komponente innerhalb eines Systems und seiner äußeren Umgebung müssen synthetisierte Implementierungen die Schnittstelle der Komponente berücksichtigen. Beispielsweise kann eine Komponente ihre Umgebung nur über wenige, unpräzise Sensoren beobachten. Darüber hinaus haben präzise Modelle einer Umgebung einer Komponente normalerweise einen unendlichen Zustandsraum, z.B. durch die Modellierung von Realzeit oder durch unbegrenzte Kommunikationspuffer. Diese Dissertation stellt neuartige Syntheseverfahren für Modelle mit unendlichem Zustandsraum vor, die die Einschränkungen berücksichtigen, die durch die Schnittstelle der synthetisierten Systemkomponenten gegeben sind. Das grundlegende Berechnungsmodell sind Spiele mit zwei Spielern und einem unendlichen Zustandsraum. Der Beitrag der Dissertation ist in drei Teile gegliedert. Der erste Teil der Dissertation liefert Entscheidbarkeitsresultate für eine Klasse von Spielen, in der der Spieler, der die Systemkomponente repräsentiert, eine endliche Menge von Beobachtungen und Aktionen hat. Ein prominenter Repräsentant dieser Klasse sind Lossy Channel Systeme. Es werden symbolische Algorithmen zur Strategiesynthese für Lossy Channel Spiele unter unvollständiger Information mit Sicherheits und Erreichbarkeits-Gewinnzielen präsentiert. Der zweite Beitrag besteht aus einem Gegenbeispiel-geführten Abstraktionsverfeinerungs-Schema zum Lösen von Spielen mit unendlichem Zustandsraum unter unvollständiger Information, in denen die Komponente endlich viele Aktionen hat aber keine endliche Menge von möglichen Beobachtungen gegeben ist. Diese Situation ist weit verbreitet z.B. bei der Synthese von Mutex-Protokollen oder Robotersteuerungen. In diesem Kontext entsprechen die Beobachtungen Beobachtungsprädikaten, die durch logische Formeln repräsentiert sind, wobei deren Berechnung ein integraler Bestandteil des Syntheseverfahrens ist. Das resultierende Verfahren kann zum Lösen von Spielen benutzt werden, die mit keiner verfügbaren Technik gelöst werden können. Letztlich werden Systeme untersucht, in denen die Komponente unendlich viele Beobachtungsprädikate hat und zwischen unendlich vielen Aktionen auswählen kann. Gezeitete Spiele unter unvollständiger Information sind eine grundlegende Klasse von Spielen, bei denen dies der Fall ist. Wir erweitern das Abstraktionsverfeinerungs-Schema, um die erste systematische Methode zur Synthese von Beobachtungsprädikaten für gezeitete Controller zu entwickeln. Es wird demonstriert, dass eine Verfeinerung der Beobachtungen, basierend auf Gegenbeispielen, ein höheres Potential aufzeigt als eine Brute-Force-Aufzählung der Beobachtungen, insbesondere für Systeme, bei denen eine feine Granularität der Beobachtungen notwendig ist