80 research outputs found
nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Models
A rigorous formalization of desired system requirements is indispensable when performing any verification task. This often limits the application of verification techniques, as writing formal specifications is an error-prone and time-consuming manual task. To facilitate this, we present nl2spec, a framework for applying Large Language Models (LLMs) to derive formal specifications (in temporal logics) from unstructured natural language. In particular, we introduce a new methodology to detect and resolve the inherent ambiguity of system requirements in natural language: we utilize LLMs to map subformulas of the formalization back to the corresponding natural language fragments of the input. Users iteratively add, delete, and edit these sub-translations to amend erroneous formalizations, which is easier than manually redrafting the entire formalization. The framework is agnostic to specific application domains and can be extended to similar specification languages and new neural models. We perform a user study to obtain a challenging dataset, which we use to run experiments on the quality of translations. We provide an open-source implementation, including a web-based frontend
Syntactic approaches to negative results in process algebras and modal logics
Concurrency as a phenomenon is observed in most of the current computer science
trends. However the inherent complexity of analyzing the behavior of such a system
is incremented due to the many different models of concurrency, the variety of applications and architectures, as well as the wide spectrum of specification languages and demanded correctness criteria. For the scope of this thesis we focus on state based models of concurrent computation, and on modal logics as specification languages. First we study syntactically the process algebras that describe several different concurrent behaviors, by analyzing their equational theories. Here, we use well-established techniques from the equational logic of processes to older and newer setups, and then transition to the use of more general and novel methods for the syntactical analysis of models of concurrent programs and specification languages. Our main contributions are several positive and negative axiomatizability results over various process algebraic languages and equivalences, along with some complexity results over the satisfiability of multi-agent modal logic with recursion, as a specification language.Samhliða sem fyrirbæri sést í flestum núverandi tölvunarfræði stefnur. Hins vegar er eðlislægt flókið að greina hegðun slíks kerfis- tem er aukið vegna margra mismunandi gerða samhliða, fjölbreytileikans af forritum og arkitektúr, svo og breitt svið forskrifta mælikvarða og kröfðust réttmætisviðmiða. Fyrir umfang þessarar ritgerðar leggjum við áherslu á ástandsbundin líkön af samhliða útreikningum og á formlegum rökfræði sem forskrift tungumálum. Fyrst skoðum við setningafræðilega ferlialgebrurnar sem lýsa nokkrum mismunandi samhliða hegðun, með því að greina jöfnukenningar þeirra. Hér notum við rótgróin tækni mynda jöfnunarrökfræði ferla til eldri og nýrri uppsetningar, og síðan umskipti yfir í notkun almennari og nýrra aðferða fyrir setningafræðileg greining á líkönum samhliða forrita og forskriftartungumála. Helstu framlög okkar eru nokkrar jákvæðar og neikvæðar niðurstöður um axiomatizability yfir ýmis ferli algebrumál og jafngildi, ásamt nokkrum samSveigjanleiki leiðir af því að fullnægjanleiki fjölþátta formrökfræði með endurkomu, sem a forskrift tungumál.RANNIS: `Open Problems in the Equational Logic of Processes’ (OPEL) (grant No 196050-051)
Reykjavik University research fund: `Runtime and Equational Verification of Concurrent Programs' (ReVoCoP) (grant No 222021
Verbesserrung der Datenflussüberwachung für Datennutzungskontrollsysteme
This thesis provides a new, hybrid approach in the field of Distributed Data Usage Control (DUC), to track the flow of data inside applications. A combination between static information flow analysis and dynamic data flow tracking enables to track selectively only those program locations that are actually relevant for a flow of data. This ensures the portability of a monitored application with low performance overhead. Beyond that, DUC systems benefit from the present approach as it reduces overapproximation in data flow tracking, and thus, provides a more precise result to enforce data usage restrictions.Diese Thesis liefert einen neuartigen hybriden Ansatz auf dem Gebiet von Distributed Data Usage Control (DUC), um den Datenfluss innerhalb einer Anwendung zu überwachen. Eine Kombination aus statischer Informationsflussanalyse und dynamischer Datenflussüberwachung ermöglicht die selektive, modulare Überwachung derjenigen Programmstellen, welche tatsächlich relevant für einen Datenfluss sind. Dadurch wird die Portabilität einer zu überwachenden Anwendung, bei geringem Performance Overhead, sichergestellt. DUC Systeme profitieren vom vorliegenden Ansatz vor allem dadurch, dass Überapproximation bei der Datenflussüberwachung reduziert wird, und somit ein präziseres Ergebnis für die Durchsetzung von Datennutzungsrestriktionen vorliegt
From LTL to rLTL monitoring
Runtime monitoring is commonly used to detect the violation of desired properties in safety critical systems by observing run prefixes of the system. Bauer et al. introduced an influential framework for monitoring Linear Temporal Logic (LTL) properties, which is based on a three-valued semantics: the formula is already satisfied by the given prefix, it is already violated, or it is still undetermined, i.e., it can be satisfied and violated. However, a wide range of formulas are not monitorable under this approach, meaning that every prefix is undetermined. In particular, Bauer et al. report that 44% of the formulas they consider in their experiments fall into this category. Recently, robust semantics for LTL were introduced to capture degrees of violation of universal properties. Here, we define robust semantics for run prefixes and show its potential in monitoring: every formula considered by Bauer et al. is monitorable under our approach. Furthermore, we show that properties expressed with the robust semantics can be monitored by deterministic automata
Combining Model Checking and Testing
Abstract Model checking and testing have a lot in common. Over the last two decades, significant progress has been made on how to broaden the scope of model checking from finite-state abstractions to actual software implementations. One way to do this consists of adapting model checking into a form of systematic testing that is applicable to industrial-size software. This chapter presents an overview of this strand of software model checking
Predictive Runtime Verification of Stochastic Systems
Runtime Verification (RV) is the formal analysis of the execution of a system against some
properties at runtime. RV is particularly useful for stochastic systems that have a non-zero
probability of failure at runtime. The standard RV assumes constructing a monitor that
checks only the currently observed execution of the system against the given properties.
This dissertation proposes a framework for predictive RV, where the monitor instead
checks the current execution with its finite extensions against some property. The extensions are generated using a prediction model, that is built based on execution samples
randomly generated from the system. The thesis statement is that predictive RV for
stochastic systems is feasible, effective, and useful.
The feasibility is demonstrated by providing a framework, called Prevent, that builds a
predictive monitor by using trained prediction models to finitely extend an execution path,
and computing the probabilities of the extensions that satisfy or violate the given property.
The prediction model is trained using statistical learning techniques from independent and
identically distributed samples of system executions. The prediction is the result of a
quantitative bounded reachability analysis on the product of the prediction model and the
automaton specifying the property. The analysis results are computed offline and stored in
a lookup table. At runtime the monitor obtains the state of the system on the prediction
model based on the observed execution, directly or by approximation, and uses the lookup
table to retrieve the computed probability that the system at the current state will satisfy
or violate the given property within some finite number of steps.
The effectiveness of Prevent is shown by applying abstraction when constructing the
prediction model. The abstraction is on the observation space based on extracting the
symmetry relation between symbols that have similar probabilities to satisfy a property.
The abstraction may introduce nondeterminism in the final model, which is handled by
using a hidden state variable when building the prediction model. We also demonstrate
that, under the convergence conditions of the learning algorithms, the prediction results
from the abstract models are the same as the concrete models.
Finally, the usefulness of Prevent is indicated in real-world applications by showing
how it can be applied for predicting rare properties, properties with very low but non-zero
probability of satisfaction. More specifically, we adjust the training algorithm that uses
the samples generated by importance sampling to generate the prediction models for rare
properties without increasing the number of samples and without having a negative impact
on the prediction accuracy
The early bird catches the worm: First verify, then monitor!
© 2018 Elsevier B.V. Trace expressions are a compact and expressive formalism, initially devised for runtime verification of agent interactions in multiagent systems, which has been successfully employed to model real-world protocols, and to generate monitors for mainstream multiagent system platforms, and generalized to support runtime verification of different kinds of properties and systems. In this paper, we propose an algorithm to check Linear Temporal Logic (LTL) properties satisfiability on trace expressions. To do this, we show how to translate a trace expression into a Büchi Automaton in order to realize an Automata-Based Model Checking. We show that this translation generates an over-approximation of our trace expression leading us to obtain a sound procedure to verify LTL properties. Once we have statically checked a set of LTL properties, we can conclude that: (1) our trace expression is formally correct (2) since we use this trace expression to generate monitors checking the runtime behavior of the system, the LTL properties verified by this trace expression are also verified by the monitored system
Dynamic contracts for verification and enforcement of real-time systems properties
Programa de Doutoramento em Informática (MAP-i) das Universidades do Minho, de Aveiro e do PortoRuntime veri cation is an emerging discipline that investigates methods and tools to enable
the veri cation of program properties during the execution of the application. The goal is
to complement static analysis approaches, in particular when static veri cation leads to
the explosion of states. Non-functional properties, such as the ones present in real-time
systems are an ideal target for this kind of veri cation methodology, as are usually out of
the range of the power and expressiveness of classic static analyses.
Current real-time embedded systems development frameworks lack support for the veri -
cation of properties using explicit time where counting time (i.e., durations) may play an
important role in the development process. Temporal logics targeting real-time systems
are traditionally undecidable. Based on a restricted fragment of Metric temporal logic with
durations (MTL-R), we present the proposed synthesis mechanisms 1) for target systems
as runtime monitors and 2) for SMT solvers as a way to get, respectively, a verdict at
runtime and a schedulability problem to be solved before execution. The later is able to
solve partially the schedulability analysis for periodic resource models and xed priority
scheduler algorithms. A domain speci c language is also proposed in order to describe
such schedulability analysis problems in a more high level way.
Finally, we validate both approaches, the rst using empirical scheduling scenarios for unimulti-
processor settings, and the second using the use case of the lightweight autopilot
system Px4/Ardupilot widely used for industrial and entertainment purposes. The former
also shows that certain classes of real-time scheduling problems can be solved, even though
without scaling well. The later shows that for the cases where the former cannot be used,
the proposed synthesis technique for monitors is well applicable in a real world scenario
such as an embedded autopilot
ight stack.A verificação do tempo de execução e uma disciplina emergente que investiga métodos e ferramentas para permitir a verificação de propriedades do programa durante a execução da aplicação. O objetivo é complementar abordagens de analise estática, em particular quando a verificação estática se traduz em explosão de estados. As propriedades não funcionais, como as que estão presentes em sistemas em tempo real, são um alvo ideal para este tipo de metodologia de verificação, como geralmente estão fora do alcance do poder e expressividade das análises estáticas clássicas.
As atuais estruturas de desenvolvimento de sistemas embebidos para tempo real não possuem suporte para a verificação de propriedades usando o tempo explicito onde a contagem de tempo (ou seja, durações) pode desempenhar um papel importante no processo de desenvolvimento. As logicas temporais que visam sistemas de tempo real são tradicionalmente indecidíveis. Com base num fragmento restrito de MTL-R (metric temporal logic with durations), apresentaremos os mecanismos de síntese 1) para sistemas alvo como monitores de tempo de execução e 2) para solvers SMT como forma de obter, respetivamente, um veredicto em tempo de execução e um problema de escalonamento para ser resolvido antes da execução. O ultimo é capaz de resolver parcialmente a analise de escalonamento para modelos de recursos periódicos e ainda para algoritmos de escalonamento de prioridade fixa. Propomos também uma linguagem especifica de domínio para descrever esses mesmos problemas de analise de escalonamento de forma mais geral e sucinta.
Finalmente, validamos ambas as abordagens, a primeira usando cenários de escalonamento empírico para sistemas uni- multi-processador e a segunda usando o caso de uso do sistema de piloto automático leve Px4/Ardupilot amplamente utilizado para fins industriais e de entretenimento. O primeiro mostra que certas classes de problemas de escalonamento em tempo real podem ser solucionadas, embora não seja escalável. O ultimo mostra que, para os casos em que a primeira opção não possa ser usada, a técnica de síntese proposta para monitores aplica-se num cenário real, como uma pilha de voo de um piloto automático embebido.This thesis was partially supported by National Funds through FCT/MEC (Portuguese
Foundation for Science and Technology) and co- nanced by ERDF (European Regional
Development Fund) under the PT2020 Partnership, within the CISTER Research Unit
(CEC/04234); FCOMP-01-0124-FEDER-015006 (VIPCORE) and FCOMP-01-0124-FEDER-
020486 (AVIACC); also by FCT and EU ARTEMIS JU, within project ARTEMIS/0003/2012,
JU grant nr. 333053 (CONCERTO); and by FCT/MEC and the EU ARTEMIS JU within
project ARTEMIS/0001/2013 - JU grant nr. 621429 (EMC2)
- …