Specification sketching for Linear Temporal Logic

Virtually all verification and synthesis techniques assume that the formal specifications are readily available, functionally correct, and fully match the engineer's understanding of the given system. However, this assumption is often unrealistic in practice: formalizing system requirements is notoriously difficult, error-prone, and requires substantial training. To alleviate this severe hurdle, we propose a fundamentally novel approach to writing formal specifications, named specification sketching for Linear Temporal Logic (LTL). The key idea is that an engineer can provide a partial LTL formula, called an LTL sketch, where parts that are hard to formalize can be left out. Given a set of examples describing system behaviors that the specification should or should not allow, the task of a so-called sketching algorithm is then to complete a given sketch such that the resulting LTL formula is consistent with the examples. We show that deciding whether a sketch can be completed falls into the complexity class NP and present two SAT-based sketching algorithms. We also demonstrate that sketching is a practical approach to writing formal specifications using a prototype implementation

Robust, Expressive, and Quantitative Linear Temporal Logics: Pick any Two for Free

Linear Temporal Logic (LTL) is the standard specification language for reactive systems and is successfully applied in industrial settings. However, many shortcomings of LTL have been identified in the literature, among them the limited expressiveness, the lack of quantitative features, and the inability to express robustness. There is work on overcoming these shortcomings, but each of these is typically addressed in isolation. This is insufficient for applications where all shortcomings manifest themselves simultaneously. Here, we tackle this issue by introducing logics that address more than one shortcoming. To this end, we combine the logics Linear Dynamic Logic, Prompt-LTL, and robust LTL, each addressing one aspect, to new logics. For all combinations of two aspects, the resulting logic has the same desirable algorithmic properties as plain LTL. In particular, the highly efficient algorithmic backends that have been developed for LTL are also applicable to these new logics. Finally, we discuss how to address all three aspects simultaneously

A Review of Formal Methods applied to Machine Learning

We review state-of-the-art formal methods applied to the emerging field of the verification of machine learning systems. Formal methods can provide rigorous correctness guarantees on hardware and software systems. Thanks to the availability of mature tools, their use is well established in the industry, and in particular to check safety-critical applications as they undergo a stringent certification process. As machine learning is becoming more popular, machine-learned components are now considered for inclusion in critical systems. This raises the question of their safety and their verification. Yet, established formal methods are limited to classic, i.e. non machine-learned software. Applying formal methods to verify systems that include machine learning has only been considered recently and poses novel challenges in soundness, precision, and scalability. We first recall established formal methods and their current use in an exemplar safety-critical field, avionic software, with a focus on abstract interpretation based techniques as they provide a high level of scalability. This provides a golden standard and sets high expectations for machine learning verification. We then provide a comprehensive and detailed review of the formal methods developed so far for machine learning, highlighting their strengths and limitations. The large majority of them verify trained neural networks and employ either SMT, optimization, or abstract interpretation techniques. We also discuss methods for support vector machines and decision tree ensembles, as well as methods targeting training and data preparation, which are critical but often neglected aspects of machine learning. Finally, we offer perspectives for future research directions towards the formal verification of machine learning systems

FlexiMC Framework: framework flexible para model checking

Los algoritmos de verificación de modelos (Model Checking) verifican una fórmula lógica sobre un modelo. Esta técnica permite tanto comprobar un correcto funcionamiento como descubrir errores de diseño y es aplicable a una gran variedad de campos. Una desventaja común al utilizar esta técnica es que dichos modelos deben ser traducidos al lenguaje concreto del sistema de Model Checking. El framework que aquí se presenta permite adaptar la técnica a los lenguajes, soportando la definición y uso de lenguajes específicos del dominio de forma directa, además de otras facilidades como permitir una visualización de contraejemplos personalizada o una gran flexibilidad en los algoritmos a usar. [ABSTRACT] Model Checking algorithms verify whether a logical formula holds in a model. Such technique allows checking the correct functionality of the modeled system as well as finding design mistakes, and can be applied in a wide variety of fields. A typical disadvantage of using this technique is the need to translate such models to the concrete Model Checking system language. The framework presented here allows adapting the technique to different languages, supporting the definition and use of domain specific languages directly. It also incorporates some other facilities like the creation of customized counter-example viewers or allowing the replacement of the specific algoritms used in the checking.Model Checking algorithms verify whether a logical formula holds in a model. Such technique allows checking the correct functionality of the modeled system as well as finding design mistakes, and can be applied in a wide variety of fields. A typical disadvantage of using this technique is the need to translate such models to the concrete Model Checking system language. The framework presented here allows adapting the technique to different languages, supporting the definition and use of domain specific languages directly. It also incorporates some other facilities like the creation of customized counter-example viewers or allowing the replacement of the specific algoritms used in the checking

Extending programs with debug-related features, with application to hardware development

The capacity and programmability of reconfigurable hardware such as FPGAs has improved steadily over the years, but they do not readily provide any mechanisms for monitoring or debugging running programs. Such mechanisms need to be written into the program itself. This is done using ad hoc methods and primitive tools when compared to CPU programming. This complicates the programming and debugging of reconfigurable hardware. We introduce Program-hosted Directability (PhD), the extension of programs to interpret direction commands at runtime to enable debugging, monitoring and profiling. Normally in hardware development such features are fixed at compile time. We present a language of directing commands, specify its semantics in terms of a simple controller that is embedded with programs, and implement a prototype for directing network programs running in hardware. We show that this approach affords significant flexibility with low impact on hardware utilisation and performance.This work has received funding from the EPSRC NaaS grant EP/K034723/1, European Union's Horizon 2020 research and innovation programme 2014-2018 under the SSICLOPS (grant agreement No. 644866), the Leverhulme Trust Early Career Fellowship ECF-2016-289 and the Newton Trust

Robust, expressive, and quantitative linear temporal logics: Pick any two for free

Linear Temporal Logic (LTL) is the standard specification language for reactive systems and is successfully applied in industrial settings. However, many shortcomings of LTL have been identified, including limited expressiveness, the lack of quantitative features, and the inability to express robustness. While there is work on overcoming these shortcomings, each of these is typically addressed in isolation, which is insufficient for any application in which all shortcomings manifest themselves simultaneously. Here, we tackle this issue by introducing logics that address more than one shortcoming. To this end, we combine Linear Dynamic Logic, Prompt-LTL, and robust LTL, each addressing one aspect, to new logics. The resulting logics have the same desirable algorithmic properties as plain LTL for all combinations of two aspects. In particular, the highly efficient algorithmic backends developed for LTL are also applicable to these new logics. Finally, we discuss how to address all three aspects simultaneously

Formal verification of a real-time operating system

Errors caused by the interaction of computer systems with the physical world are hard to mitigate but errors related to the underlying software can be prevented by a more rigorous development of software code. In the context of critical systems, a failure caused by software errors could lead to consequences that are determined to be unacceptable. At the heart of a critical system, a real-time operating system is commonly found. Since the reliability of the entire system depends upon having a reliable operating system, verifying that the operating systems functions as desired is of prime interest. One solution to verify the correctness of significant properties of an existing real-time operating system microkernel (FreeRTOS) applies assisted proof checking to its formalized specification description. The experiment consists of describing real-time operating system characteristics, such as memory safety and scheduler determinism, in Separation Logic — a formal language that allows reasoning about the behaviour of the system in terms of preconditions and postconditions. Once the desired properties are defined in a formal language, a theorem can be constructed to describe the validity of such formula for the given FreeRTOS implementation. Then, by using the Coq proof assistant, a machine-checked proof that such properties hold for FreeRTOS can be carried out. By expressing safety and deterministic properties of an existing real-time operating systems and proving them correct we demonstrate that the current state-of-the-art in theorem-based formal verification, including appropriate logics and proof assistants, make it possible to provide a machine-checked proof of the specification of significant properties for FreeRTOS