Symbolic Controller Synthesis for B\"uchi Specifications on Stochastic Systems

We consider the policy synthesis problem for continuous-state controlled Markov processes evolving in discrete time, when the specification is given as a B\"uchi condition (visit a set of states infinitely often). We decompose computation of the maximal probability of satisfying the B\"uchi condition into two steps. The first step is to compute the maximal qualitative winning set, from where the B\"uchi condition can be enforced with probability one. The second step is to find the maximal probability of reaching the already computed qualitative winning set. In contrast with finite-state models, we show that such a computation only gives a lower bound on the maximal probability where the gap can be non-zero. In this paper we focus on approximating the qualitative winning set, while pointing out that the existing approaches for unbounded reachability computation can solve the second step. We provide an abstraction-based technique to approximate the qualitative winning set by simultaneously using an over- and under-approximation of the probabilistic transition relation. Since we are interested in qualitative properties, the abstraction is non-probabilistic; instead, the probabilistic transitions are assumed to be under the control of a (fair) adversary. Thus, we reduce the original policy synthesis problem to a B\"uchi game under a fairness assumption and characterize upper and lower bounds on winning sets as nested fixed point expressions in the $\mu$-calculus. This characterization immediately provides a symbolic algorithm scheme. Further, a winning strategy computed on the abstract game can be refined to a policy on the controlled Markov process. We describe a concrete abstraction procedure and demonstrate our algorithm on two case studies

Provably-Correct Task Planning for Autonomous Outdoor Robots

Autonomous outdoor robots should be able to accomplish complex tasks safely and reliably while considering constraints that arise from both the environment and the physical platform. Such tasks extend basic navigation capabilities to specify a sequence of events over time. For example, an autonomous aerial vehicle can be given a surveillance task with contingency plans while complying with rules in regulated airspace, or an autonomous ground robot may need to guarantee a given probability of success while searching for the quickest way to complete the mission. A promising approach for the automatic synthesis of trusted controllers for complex tasks is to employ techniques from formal methods. In formal methods, tasks are formally specified symbolically with temporal logic. The robot then synthesises a controller automatically to execute trusted behaviour that guarantees the satisfaction of specified tasks and regulations. However, a difficulty arises from the lack of expressivity, which means the constraints affecting outdoor robots cannot be specified naturally with temporal logic. The goal of this thesis is to extend the capabilities of formal methods to express the constraints that arise from outdoor applications and synthesise provably-correct controllers with trusted behaviours over time. This thesis focuses on two important types of constraints, resource and safety constraints, and presents three novel algorithms that express tasks with these constraints and synthesise controllers that satisfy the specification. Firstly, this thesis proposes an extension to probabilistic computation tree logic (PCTL) called resource threshold PCTL (RT-PCTL) that naturally defines the mission specification with continuous resource threshold constraints; furthermore, it synthesises an optimal control policy with respect to the probability of success. With RT-PCTL, a state with accumulated resource out of the specified bound is considered to be failed or saturated depending on the specification. The requirements on resource bounds are naturally encoded in the symbolic specification, followed by the automatic synthesis of an optimal controller with respect to the probability of success. Secondly, the thesis proposes an online algorithm called greedy Buchi algorithm (GBA) that reduces the synthesis problem size to avoid the scalability problem. A framework is then presented with realistic control dynamics and physical assumptions in the environment such as wind estimation and fuel constraints. The time and space complexity for the framework is polynomial in the size of the system state, which is efficient for online synthesis. Lastly, the thesis proposes a synthesis algorithm for an optimal controller with respect to completion time given the minimum safety constraints. The algorithm naturally balances between completion time and safety. This work proves an analytical relationship between the probability of success and the conditional completion time given the mission specification. The theoretical contributions in this thesis are validated through realistic simulation examples. This thesis identifies and solves two core problems that contribute to the overall vision of developing a theoretical basis for trusted behaviour in outdoor robots. These contributions serve as a foundation for further research in multi-constrained task planning where a number of different constraints are considered simultaneously within a single framework

Formal analysis of control systems via inductive approaches

This dissertation is concerned with the formal analysis of complex control systems via inductive approaches using barrier certificates. In general, safety-critical applications such as air traffic networks, autonomous vehicles, power grids, medical devices, and robotic equipment, are expected to satisfy complex logic specifications including but not limited to safety, reachability, and security. Due to several factors such as the continuous-state evolution of systems' trajectories, large systems' sizes, disturbances, etc., verification and synthesis of control systems against such high-level logic specifications is a challenging task. An interesting yet simple way to tackle the verification and synthesis problem for logic specifications is to utilize inductive approaches based on barrier certificates. Barrier certificates take the form of inductive invariants and provide sufficient conditions for the satisfaction of safety or reachability specifications. Therefore, the verification and synthesis problem is reduced to the discovery of suitable barrier certificates. However, finding suitable barrier certificates can be a difficult problem due to several factors. First, the computation of barrier certificates is not scalable to large-scale systems. Second, the conditions imposed by barrier certificates are restrictive, making it difficult to search for one. Third, barrier certificate-based methods are limited to the analysis of safety or reachability specifications. As a result, they are not directly applicable to complicated logic tasks such as those expressed by omega-regular properties or (in)finite strings over automata, as well as security specifications such as those expressed by hyperproperties. In this regard, the dissertation focuses on alleviating the aforementioned issues and provides novel techniques to verify and synthesize controllers for (possibly large-scale and stochastic) control systems against the aforementioned specifications. The first part of the thesis proposes a compositional framework for scalable construction of control barrier certificates for large-scale discrete-time stochastic control systems. In particular, we show that by considering the large-scale system as an interconnected one composed of several subsystems, one may construct control barrier certificates for the interconnected system by searching for so-called control sub-barrier certificates for subsystems and utilizing some compositionality conditions based on small-gain and dissipativity approaches. Correspondingly, one may also synthesize controllers that can be applied to the interconnected system in a decentralized manner, so that the large-scale system satisfies safety specifications over (in)finite time horizons with some probability lower bounds. In the second part of the thesis, we propose a new notion of k-inductive barrier certificates for the verification of (stochastic) discrete-time dynamical systems against safety and reachability specifications. In particular, we illustrate that due to the restrictive nature of the traditional barrier certificate conditions, it is not always possible to find suitable barrier certificates even when the system is guaranteed to satisfy the desired specifications. Then, we extend the k-induction principle utilized in software verification to propose several notions of k-inductive barrier certificates that relax the traditional barrier certificate conditions. As a result, larger classes of functions may act as barrier certificates, making them easier to find. In the context of non-stochastic systems, we propose two notions of k-inductive barrier certificates and provide formal guarantees for safety specifications. In the case of stochastic systems, we propose one notion of k-inductive barrier certificates for safety and two notions for tackling reachability specifications. Then, we obtain probabilistic guarantees for the satisfaction of safety and reachability specifications over infinite time horizons, respectively. The last part of the thesis is concerned with the analysis of (stochastic) control systems against complex logic specifications beyond safety and reachability. First, we consider the synthesis problem for (possibly large-scale) stochastic control systems against \emph{trace properties}, which describe specifications over individual traces of the system. Examples of such properties include omega-regular languages or (in)finite words over automata. We provide an automata-theoretic approach to decompose such complex specifications into sequential safety specifications. We then utilize the probability guarantees obtained for the safety specifications and combine them to obtain probability lower bounds for the satisfaction of original specifications. We provide such guarantees over both finite and infinite time horizons. Secondly, we consider the verification problem for non-stochastic systems against specifications that can be expressed over sets of traces, called hyperproperties. Hyperproperties can express many security and planning specifications that cannot be considered using omega-regular languages. In this context, we provide an automata-theoretic approach to decompose hyperproperties into smaller verification conditions called conditional invariances. Then, we introduce a new notion of so-called augmented barrier certificates constructed on the augmented system (i.e., self-composition of the system) to provide guarantees for the satisfaction of the conditional invariances. These guarantees may then be combined to achieve the satisfaction of original hyperproperties.Diese Dissertation befasst sich mit der formalen Analyse komplexer Regelkreise, wobei induktive Ansätze unter Verwendung von Barrierezertifikaten zum Einsatz kommen. Im Allgemeinen wird von sicherheitskritischen Anwendungen wie Flugverkehrsnetzen, autonomen Fahrzeugen, Stromnetzen, medizinischen Geräten und Roboteranlagen erwartet, dass sie komplexe formale Spezifikationen erfüllen, um etwa die Betriebs- und Informationssicherheit zu gewährleisten oder anderweitige Ziele wie beispielsweise bestimmte Erreichbarkeitseigenschaften einzuhalten. Aufgrund verschiedener Faktoren wie der kontinuierlichen Zeitentwicklung der Systemtrajektorien, der Größe der Systeme, Störungen usw. ist die Verifizierung und Synthese von Steuersystemen anhand von solch allgemeinen logischen Spezifikationen eine anspruchsvolle Aufgabe. Ein interessanter und dennoch einfacher Weg, das Verifikations- und Syntheseproblem für formale Spezifikationen anzugehen, ist die Verwendung von induktiven Ansäzen, die auf Barrierezertifikaten basieren. Diese haben die Form von induktiven Invarianten und liefern hinreichende Bedingungen für die Erfüllung von Sicherheits- oder Erreichbarkeitsanforderungen. Daher reduziert sich das Verifikations- und Syntheseproblem auf die Entdeckung geeigneter Barrierezertifikate. Die Suche nach geeigneten Barrierezertifikaten kann jedoch aufgrund mehrerer Faktoren ein schwieriges Problem darstellen. Erstens skaliert die Berechnung von Barrierezertifikaten nicht ohne weiteres auf große Systeme. Zweitens stellen die von Barrierezertifikaten auferlegten Bedingungen eine starke Einschränkung dar, was die Suche nach einem solchen Zertifikat erschwert. Drittens sind auf Barrierezertifikaten basierende Methoden auf die Analyse von Sicherheits- oder Erreichbarkeitsspezifikationen limitiert. Infolgedessen sind sie nicht direkt auf komplizierte logische Aufgaben anwendbar, wie z.B. solche, die durch omega-reguläre Eigenschaften oder (un)endliche Zeichenketten über Automaten ausgedrückt werden, oder auf bestimmte Sicherheitsspezifikationen, etwa wenn sie durch Hypereigenschaften ausgedrückt werden. In dieser Hinsicht konzentriert sich die Dissertation auf die Linderung der oben genannten Probleme und stellt neuartige Techniken zur Verifikation und Synthese von Reglern f\ür (möglicherweise große und stochastische) Regelkreise hinsichtlich der oben genannten Spezifikationen zur Verfügung. IIm ersten Teil der Dissertation wird ein kompositorischer Rahmen für die skalierbare Konstruktion von Regelungsbarrierezertifikaten für große, zeitdiskrete und stochastische Regelkreise vorgeschlagen. Insbesondere zeigen wir, dass man durch die Zerlegung des Systems in mehrere, zusammenhängende Subsysteme Regelungsbarrierezertifikate für das Gesamtsystem konstruieren kann, indem man nach sogenannten Regelungsunterbarrierezertifikaten für die Subsysteme sucht. Hierzu lassen sich bestimmte Kompositionalitätsbedingungen auf der Basis von small-gain und dissipativity Ansätzen formulieren. Dementsprechend kann man auch Regler synthetisieren, die dezentral auf die zusammenhängenden Komponenten des Systems angewendet werden können, so dass das Gesamtsystem die Sicherheitsspezifikationen über (un)endliche Zeithorizonte mit einigen Wahrscheinlichkeitsuntergrenzen erfüllt. Im zweiten Teil der Dissertation stellen wir das neuartige Konzept der k-induktiven Barrierezertifikate für die Verifikation von (stochastischen) zeitdiskreten dynamischen Systemen bezüglich Sicherheits- und Erreichbarkeitsspezifikationen vor. Insbesondere zeigen wir, dass es aufgrund der restriktiven Natur der traditionellen Barrierezertifikatsbedingungen nicht immer möglich ist, geeignete Barrierezertifikate zu finden, selbst wenn das System garantiert die gewünschten Spezifikationen erfüllt. Im Anschluss erweitern wir das k-Induktionsprinzip, das in der Softwareverifikation verwendet wird, indem wir mehrere Konzepte für k-induktive Barrierezertifikate vorschlagen, die die traditionellen Barrierezertifikatsbedingungen lockern. Infolgedessen können größere Klassen von Funktionen als Barrierezertifikate fungieren, wodurch sie leichter zu finden sind. Im Zusammenhang mit nicht-stochastischen Systemen führen wir zwei Ausprägungen von k-induktiven Barrierezertifikaten ein und geben formale Garantien für Sicherheitsspezifikationen. Für stochastische Systeme stellen wir ein Konzept für k-induktive Barrierezertifikate für Sicherheit und zwei Konzepte für die Behandlung von Erreichbarkeitsspezifikationen vor. Danach erarbeiten wir probabilistische Garantien für die Erfüllung von Sicherheits- und Erreichbarkeitsanforderungen über unendliche Zeithorizonte. Der letzte Teil der Dissertation befasst sich mit der Analyse von (stochastischen) Regelkreisen bezüglich komplexer logischer Spezifikationen jenseits von Sicherheit und Erreichbarkeit. Zunächst betrachten wir das Syntheseproblem für (möglicherweise großräumige) stochastische Regelkreise im Hinblick auf Spureigenschaften, also Spezifikationen über einzelne Spuren des Systems. Beispiele für solche Eigenschaften sind omega-reguläre Sprachen oder (un)endliche Wörter über Automaten. Wir bieten einen automaten-theoretischen Ansatz, um solche komplexen Spezifikationen in sequenzielle Sicherheitsspezifikationen zu zerlegen. Wir verwenden sogleich die für die Sicherheitsspezifikationen erhaltenen Wahrscheinlichkeitsgarantien und kombinieren sie, um untere Wahrscheinlichkeitsschranken für die Erfüllung der ursprünglichen Spezifikationen zu erhalten. Wir geben derartige Garantien sowohl für endliche als auch für unendliche Zeithorizonte. Im Anschluss betrachten wir das Verifikationsproblem für nicht-stochastische Systeme bezüglich Spezifikationen, die über Mengen von Spuren, so genannte Hypereigenschaften, ausgedrückt werden können. Hypereigenschaften können viele Sicherheits- und Planungsspezifikationen ausdrücken, die mittels omega-regulären Sprachen nicht betrachtet werden können. In diesem Zusammenhang stellen wir einen automaten-theoretischen Ansatz zur Verfügung, um Hypereigenschaften in kleinere Verifikationsbedingungen, sogenannte bedingten Invarianzen, zu zerlegen. Darauf aufbauend führen wir das Konzept des erweiterten Barrierezertifikats ein, welches auf dem (mittels Selbstkomposition) erweiterten System konstruiert wird, um Garantien für die Erfüllung der bedingten Invarianzen zu geben. Diese Garantien können dann wiederum kombiniert werden, um die Erfüllung der ursprünglichen Hypereigenschaften zu erreichen

Fast Symbolic Algorithms for Omega-Regular Games under Strong Transition Fairness

We consider fixpoint algorithms for two-player games on graphs with $\omega$-regular winning conditions, where the environment is constrained by a strong transition fairness assumption. Strong transition fairness is a widely occurring special case of strong fairness, which requires that any execution is strongly fair with respect to a specified set of live edges: whenever the source vertex of a live edge is visited infinitely often along a play, the edge itself is traversed infinitely often along the play as well. We show that, surprisingly, strong transition fairness retains the algorithmic characteristics of the fixpoint algorithms for $\omega$-regular games -- the new algorithms can be obtained simply by replacing certain occurrences of the controllable predecessor by a new almost sure predecessor operator. For Rabin games with $k$ pairs, the complexity of the new algorithm is $O(n^{k+2}k!)$ symbolic steps, which is independent of the number of live edges in the strong transition fairness assumption. Further, we show that GR(1) specifications with strong transition fairness assumptions can be solved with a 3-nested fixpoint algorithm, same as the usual algorithm. In contrast, strong fairness necessarily requires increasing the alternation depth depending on the number of fairness assumptions. We get symbolic algorithms for (generalized) Rabin, parity and GR(1) objectives under strong transition fairness assumptions as well as a direct symbolic algorithm for qualitative winning in stochastic $\omega$-regular games that runs in $O(n^{k+2}k!)$ symbolic steps, improving the state of the art. Finally, we have implemented a BDD-based synthesis engine based on our algorithm. We show on a set of synthetic and real benchmarks that our algorithm is scalable, parallelizable, and outperforms previous algorithms by orders of magnitude

Mightyl: A compositional translation from mitl to timed automata

Metric Interval Temporal Logic (MITL) was first proposed in the early 1990s as a specification formalism for real-time systems. Apart from its appealing intuitive syntax, there are also theoretical evidences that make MITL a prime real-time counterpart of Linear Temporal Logic (LTL). Unfortunately, the tool support for MITL verification is still lacking to this day. In this paper, we propose a new construction from MITL to timed automata via very-weak one-clock alternating timed automata. Our construction subsumes the well-known construction from LTL to Büchi automata by Gastin and Oddoux and yet has the additional benefits of being compositional and integrating easily with existing tools. We implement the construction in our new tool MightyL and report on experiments using Uppaal and LTSmin as back-ends

Formal Methods for Autonomous Systems

Formal methods refer to rigorous, mathematical approaches to system development and have played a key role in establishing the correctness of safety-critical systems. The main building blocks of formal methods are models and specifications, which are analogous to behaviors and requirements in system design and give us the means to verify and synthesize system behaviors with formal guarantees. This monograph provides a survey of the current state of the art on applications of formal methods in the autonomous systems domain. We consider correct-by-construction synthesis under various formulations, including closed systems, reactive, and probabilistic settings. Beyond synthesizing systems in known environments, we address the concept of uncertainty and bound the behavior of systems that employ learning using formal methods. Further, we examine the synthesis of systems with monitoring, a mitigation technique for ensuring that once a system deviates from expected behavior, it knows a way of returning to normalcy. We also show how to overcome some limitations of formal methods themselves with learning. We conclude with future directions for formal methods in reinforcement learning, uncertainty, privacy, explainability of formal methods, and regulation and certification

Synthesizing Permissive Winning Strategy Templates for Parity Games

We present a novel method to compute \emph{permissive winning strategies} in two-player games over finite graphs with $\omega$-regular winning conditions. Given a game graph $G$ and a parity winning condition $\Phi$, we compute a \emph{winning strategy template} $\Psi$ that collects an infinite number of winning strategies for objective $\Phi$ in a concise data structure. We use this new representation of sets of winning strategies to tackle two problems arising from applications of two-player games in the context of cyber-physical system design -- (i) \emph{incremental synthesis}, i.e., adapting strategies to newly arriving, \emph{additional} $\omega$-regular objectives $\Phi'$, and (ii) \emph{fault-tolerant control}, i.e., adapting strategies to the occasional or persistent unavailability of actuators. The main features of our strategy templates -- which we utilize for solving these challenges -- are their easy computability, adaptability, and compositionality. For \emph{incremental synthesis}, we empirically show on a large set of benchmarks that our technique vastly outperforms existing approaches if the number of added specifications increases. While our method is not complete, our prototype implementation returns the full winning region in all 1400 benchmark instances, i.e., handling a large problem class efficiently in practice.Comment: CAV'2

Logical and deep learning methods for temporal reasoning

In this thesis, we study logical and deep learning methods for the temporal reasoning of reactive systems. In Part I, we determine decidability borders for the satisfiability and realizability problem of temporal hyperproperties. Temporal hyperproperties relate multiple computation traces to each other and are expressed in a temporal hyperlogic. In particular, we identify decidable fragments of the highly expressive hyperlogics HyperQPTL and HyperCTL*. As an application, we elaborate on an enforcement mechanism for temporal hyperproperties. We study explicit enforcement algorithms for specifications given as formulas in universally quantified HyperLTL. In Part II, we train a (deep) neural network on the trace generation and realizability problem of linear-time temporal logic (LTL). We consider a method to generate large amounts of additional training data from practical specification patterns. The training data is generated with classical solvers, which provide one of many possible solutions to each formula. We demonstrate that it is sufficient to train on those particular solutions such that the neural network generalizes to the semantics of the logic. The neural network can predict solutions even for formulas from benchmarks from the literature on which the classical solver timed out. Additionally, we show that it solves a significant portion of problems from the annual synthesis competition (SYNTCOMP) and even out-of-distribution examples from a recent case study.Diese Arbeit befasst sich mit logischen Methoden und mehrschichtigen Lernmethoden für das zeitabhängige Argumentieren über reaktive Systeme. In Teil I werden die Grenzen der Entscheidbarkeit des Erfüllbarkeits- und des Realisierbarkeitsproblem von temporalen Hypereigenschaften bestimmt. Temporale Hypereigenschaften setzen mehrere Berechnungsspuren zueinander in Beziehung und werden in einer temporalen Hyperlogik ausgedrückt. Insbesondere werden entscheidbare Fragmente der hochexpressiven Hyperlogiken HyperQPTL und HyperCTL* identifiziert. Als Anwendung wird ein Enforcement-Mechanismus für temporale Hypereigenschaften erarbeitet. Explizite Enforcement-Algorithmen für Spezifikationen, die als Formeln in universell quantifiziertem HyperLTL angegeben werden, werden untersucht. In Teil II wird ein (mehrschichtiges) neuronales Netz auf den Problemen der Spurgenerierung und Realisierbarkeit von Linear-zeit Temporallogik (LTL) trainiert. Es wird eine Methode betrachtet, um aus praktischen Spezifikationsmustern große Mengen zusätzlicher Trainingsdaten zu generieren. Die Trainingsdaten werden mit klassischen Solvern generiert, die zu jeder Formel nur eine von vielen möglichen Lösungen liefern. Es wird gezeigt, dass es ausreichend ist, an diesen speziellen Lösungen zu trainieren, sodass das neuronale Netz zur Semantik der Logik generalisiert. Das neuronale Netz kann Lösungen sogar für Formeln aus Benchmarks aus der Literatur vorhersagen, bei denen der klassische Solver eine Zeitüberschreitung hatte. Zusätzlich wird gezeigt, dass das neuronale Netz einen erheblichen Teil der Probleme aus dem jährlichen Synthesewettbewerb (SYNTCOMP) und sogar Beispiele außerhalb der Distribution aus einer aktuellen Fallstudie lösen kann