Stochastic transition systems: bisimulation, logic, and composition

Cyber-physical systems and the Internet of Things raise various challenges concerning the modelling and analysis of large modular systems. Models for such systems typically require uncountable state and action spaces, samplings from continuous distributions, and non-deterministic choices over uncountable many alternatives. In this thesis we fo- cus on a general modelling formalism for stochastic systems called stochastic transition system. We introduce a novel composition operator for stochastic transition systems that is based on couplings of probability measures. Couplings yield a declarative modelling paradigm appropriate for the formalisation of stochastic dependencies that are caused by the interaction of components. Congruence results for our operator with respect to standard notions for simulation and bisimulation are presented for which the challenge is to prove the existence of appropriate couplings. In this context a theory for stochastic transition systems concerning simulation, bisimulation, and trace-distribution relations is developed. We show that under generic Souslin conditions, the simulation preorder is a subset of trace-distribution inclusion and accordingly, bisimulation equivalence is finer than trace-distribution equivalence. We moreover establish characterisations of the simulation preorder and the bisimulation equivalence for a broad subclass of stochastic transition systems in terms of expressive action-based probabilistic logics and show that these characterisations are still maintained by small fragments of these logics, respectively. To treat associated measurability aspects, we rely on methods from descriptive set theory, properties of Souslin sets, as well as prominent measurable-selection principles.:1 Introduction 2 Probability measures on Polish spaces 3 Stochastic transition systems 4 Simulations and trace distributions for Souslin systems 5 Action-based probabilistic temporal logics 6 Parallel composition based on spans and couplings 7 Relations to models from the literature 8 Conclusions 9 Bibliograph

Decision algorithms for modelling, optimal control and verification of probabilistic systems

Markov Decision Processes (MDPs) constitute a mathematical framework for modelling systems featuring both probabilistic and nondeterministic behaviour. They are widely used to solve sequential decision making problems and applied successfully in operations research, arti?cial intelligence, and stochastic control theory, and have been extended conservatively to the model of probabilistic automata in the context of concurrent probabilistic systems. However, when modeling a physical system they suffer from several limitations. One of the most important is the inherent loss of precision that is introduced by measurement errors and discretization artifacts which necessarily happen due to incomplete knowledge about the system behavior. As a result, the true probability distribution for transitions is in most cases an uncertain value, determined by either external parameters or con?dence intervals. Interval Markov decision processes (IMDPs) generalize classical MDPs by having interval-valued transition probabilities. They provide a powerful modelling tool for probabilistic systems with an additional variation or uncertainty that re?ects the absence of precise knowledge concerning transition probabilities. In this dissertation, we focus on decision algorithms for modelling and performance evaluation of such probabilistic systems leveraging techniques from mathematical optimization. From a modelling viewpoint, we address probabilistic bisimulations to reduce the size of the system models while preserving the logical properties they satisfy. We also discuss the key ingredients to construct systems by composing them out of smaller components running in parallel. Furthermore, we introduce a novel stochastic model, Uncertain weighted Markov Decision Processes (UwMDPs), so as to capture quantities like preferences or priorities in a nondeterministic scenario with uncertainties. This model is close to the model of IMDPs but more convenient to work with in the context of bisimulation minimization. From a performance evaluation perspective, we consider the problem of multi-objective robust strategy synthesis for IMDPs, where the aim is to ?nd a robust strategy that guarantees the satisfaction of multiple properties at the same time in face of the transition probability uncertainty. In this respect, we discuss the computational complexity of the problem and present a value iteration-based decision algorithm to approximate the Pareto set of achievable optimal points. Moreover, we consider the problem of computing maximal/minimal reward-bounded reachability probabilities on UwMDPs, for which we present an ef?cient algorithm running in pseudo-polynomial time. We demonstrate the practical effectiveness of our proposed approaches by applying them to a collection of real-world case studies using several prototypical tools.Markov-Entscheidungsprozesse (MEPe) bilden den Rahmen für die Modellierung von Systemen, die sowohl stochastisches als auch nichtdeterministisches Verhalten beinhalten. Diese Modellklasse hat ein breites Anwendungsfeld in der Lösung sequentieller Entscheidungsprobleme und wird erfolgreich in der Operationsforschung, der künstlichen Intelligenz und in der stochastischen Kontrolltheorie eingesetzt. Im Bereich der nebenläu?gen probabilistischen Systeme wurde sie konservativ zu probabilistischen Automaten erweitert. Verwendet man MEPe jedoch zur Modellierung physikalischer Systeme so zeigt es sich, dass sie an einer Reihe von Einschränkungen leiden. Eines der schwerwiegendsten Probleme ist, dass das tatsächliche Verhalten des betrachteten Systems zumeist nicht vollständig bekannt ist. Durch Messfehler und Diskretisierungsartefakte ist ein Verlust an Genauigkeit unvermeidbar. Die tatsächlichen Übergangswahrscheinlichkeitsverteilungen des Systems sind daher in den meisten Fällen nicht exakt bekannt, sondern hängen von äußeren Faktoren ab oder können nur durch Kon?denzintervalle erfasst werden. Intervall Markov-Entscheidungsprozesse (IMEPe) verallgemeinern klassische MEPe dadurch, dass die möglichen Übergangswahrscheinlichkeitsverteilungen durch Intervalle ausgedrückt werden können. IMEPe sind daher ein mächtiges Modellierungswerkzeug für probabilistische Systeme mit unbestimmtem Verhalten, dass sich dadurch ergibt, dass das exakte Verhalten des realen Systems nicht bekannt ist. In dieser Doktorarbeit konzentrieren wir uns auf Entscheidungsverfahren für die Modellierung und die Auswertung der Eigenschaften solcher probabilistischer Systeme indem wir Methoden der mathematischen Optimierung einsetzen. Im Bereich der Modellierung betrachten wir probabilistische Bisimulation um die Größe des Systemmodells zu reduzieren während wir gleichzeitig die logischen Eigenschaften erhalten. Wir betrachten außerdem die Schlüsseltechniken um Modelle aus kleineren Komponenten, die parallel ablaufen, kompositionell zu generieren. Weiterhin führen wir eine neue Art von stochastischen Modellen ein, sogenannte Unsichere Gewichtete Markov-Entscheidungsprozesse (UgMEPe), um Eigenschaften wie Implementierungsentscheidungen und Benutzerprioritäten in einem nichtdeterministischen Szenario ausdrücken zu können. Dieses Modell ähnelt IMEPe, ist aber besser für die Minimierung bezüglich Bisimulation geeignet. Im Bereich der Auswertung von Modelleigenschaften betrachten wir das Problem, Strategien zu generieren, die in der Lage sind den Nichtdeterminismus so aufzulösen, dass mehrere gewünschte Eigenschaften gleichzeitig erfüllt werden können, wobei jede mögliche Auswahl von Wahrscheinlichkeitsverteilungen aus den Übergangsintervallen zu respektieren ist. Wir betrachten die Komplexitätsklasse dieses Problems und diskutieren einen auf Werte-Iteration beruhenden Algorithmus um die Pareto-Menge der erreichbaren optimalen Punkte anzunähern. Weiterhin betrachten wir das Problem, minimale und maximale Erreichbarkeitswahrscheinlichkeiten zu berechnen, wenn wir eine obere Grenze für dieakkumulierten Pfadkosten einhalten müssen. Für dieses Problem diskutieren wir einen ef?zienten Algorithmus mit pseudopolynomieller Zeit. Wir zeigen die Ef?zienz unserer Ansätze in der Praxis, indem wir sie prototypisch implementieren und auf eine Reihe von realistischen Fallstudien anwenden

Principles of Markov automata

A substantial amount of today's engineering problems revolve around systems that are concurrent and stochastic by their nature. Solution approaches attacking these problems often rely on the availability of formal mathematical models that reflect such systems as comprehensively as possible. In this thesis, we develop a compositional model, Markov automata, that integrates concurrency, and probabilistic and timed stochastic behaviour. This is achieved by blending two well-studied constituent models, probabilistic automata and interactive Markov chains. A range of strong and weak bisimilarity notions are introduced and evaluated as candidate relations for a natural behavioural equivalence between systems. Among them, weak distribution bisimilarity stands out as a natural notion being more oblivious to the probabilistic branching structure than prior notions. We discuss compositionality, axiomatizations, decision and minimization algorithms, state-based characterizations and normal forms for weak distribution bisimilarity. In addition, we detail how Markov automata and weak distribution bisimilarity can be employed as a semantic basis for generalized stochastic Petri nets, in such a way that known shortcomings of their classical semantics are ironed out in their entirety.Ein beträchtlicher Teil gegenwärtiger ingenieurwissenschafter Probleme erstreckt sich auf Sys- teme, die ihrer Natur nach sowohl stochastisch als auch nebenläufig sind. Lösungsansätze fußen hierbei häufig auf der Verfügbarkeit formaler mathematischer Modelle, die es erlauben, die Spez- ifika jener Systeme möglichst erschöpfend zu erfassen. In dieser Dissertation entwickeln wir ein kompositionelles Modell namens Markov-Automaten, das Nebenläufigkeit mit probabilistis- chen und stochastischen Prozessen integriert. Dies wird durch die Verschmelzung der zweier bekannter Modellklassen erreicht, und zwar die der probabilistischen Automaten und die der interaktiven Markovketten. Wir entwickeln dabei ein Spektrum verschiedener, starker und schwacher Bisimulationsrelationen und beurteilen sie im Hinblick auf ihre Eignung als natür- liche Verhaltensäquivalenz zwischen Systemen. Die schwache Wahrscheinlichkeitsverteilungs- bisimulation sticht dabei als natürliche Wahl hervor, da sie die probabilistische Verzwei- gungsstruktur treffender abstrahiert als bisher bekannte Bisimulationsrelationen. Wir betra- chten des Weiteren Kompositionalitätseigenschaften, Axiomatisierungen, Entscheidungs- und Minimierungsalgorithmen, sowie zustandsbasierte Charakterisierungen und Normalformen für die schwache Wahrscheinlichkeitsverteilungsbisimulation. Abschließend legen wir dar, dass Markov-Automaten und die schwacheWahrscheinlichkeitsverteilungsbisimulation als Grundlage für eine verbesserte Semantik von verallgemeinerten stochastischen Petrinetzen dienen kann, welche bekannte Mängel der klassischen Semantik vollständig behebt

Foundations of Software Science and Computation Structures

This open access book constitutes the proceedings of the 22nd International Conference on Foundations of Software Science and Computational Structures, FOSSACS 2019, which took place in Prague, Czech Republic, in April 2019, held as part of the European Joint Conference on Theory and Practice of Software, ETAPS 2019. The 29 papers presented in this volume were carefully reviewed and selected from 85 submissions. They deal with foundational research with a clear significance for software science

Measures on probabilistic automata

In questa tesi consideriamo i processi probabilistici non-deterministici modellati attraverso automi. Il nostro obiettivo \`e l'analisi dei problemi di bisimulazioni approssimate. Queste relazioni sono usate, generalmente, per semplificare i modelli di alcuni sistemi e per modellare agenti e attaccanti nei protocolli di sicurezza. In questo ultimo campo ci sono diversi proposte di utilizzo di metriche, le quali sono l'analogo quantitativo della bisimulazione probabilistica e permettono una miglior precisione. Una metrica \`e grossomodo un grado di similarit\`a tra stati. Iniziando dalla formalizzazione di (bi)simulazione approssimata data nel lavoro di Turrini, definiamo due metriche su stati e su distribuzioni. Queste metriche sono basate sul concetto di errore ammesso durante la simulazione di uno stato rispetto un altro stato. Investigheremo la relazione tra queste metriche con una metrica largamente utilizzata, la metrica di Kantorovich, e scopriremo che esse sono equivalenti. Poi riadatteremo per gli automi probabilistici il trasformatore di misure proposto da De Alfaro e al., ottenendo un nuovo funzionale F che \`e una estensione conservativa dei trasformatori proposti in letteratura. Mostreremo che il minimo punto fisso di F coincide con la sua sovra-approssimazione dalle misure derivate dal lavoro di Turrini, attraverso la dimostrazione dell'esistenza di una stretta relazione tra le bisimulazioni approssimate di Turrini con le metriche in letteratura.In this thesis we consider nondeterministic probabilistic processes modeled by automata. Our purpose is the analysis of the problem of approximated bisimulations. These relations are used, generally, to simplify the models of some systems and to model agents and attackers in security protocols. For the latter field there are several proposals to use metrics, which are the quantitative analogue of probabilistic bisimilarity and allow a greater precision. A metric is about a degree of similarity between states. Starting from the formalisation of approximate (bi)simulation given in Turrini's work, we define two metrics on states and on distributions. These metrics are based on the concept of error allowed during the simulation of a state with respect to another one. We investigate the relation between these metrics with a largely used one, the Kantorovich metric, and discover that they are equivalent. Then we recast for probabilistic automata the transformer of measures proposed by De Alfaro et al., obtaining a new functional F that is a conservative extension of the transformers proposed in the literature. We show that the minimum fix point of F coincides with its over-aproximated by the measures derived from Turrini's work thus showing the existence of a strict relation between the Turrini\u2019s approximate bisimulations with the literature on metrics