Learning Markov Decision Processes for Model Checking

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed system behaviors. In this paper we extend the algorithm on learning probabilistic automata to reactive systems, where the observed system behavior is in the form of alternating sequences of inputs and outputs. We propose an algorithm for automatically learning a deterministic labeled Markov decision process model from the observed behavior of a reactive system. The proposed learning algorithm is adapted from algorithms for learning deterministic probabilistic finite automata, and extended to include both probabilistic and nondeterministic transitions. The algorithm is empirically analyzed and evaluated by learning system models of slot machines. The evaluation is performed by analyzing the probabilistic linear temporal logic properties of the system as well as by analyzing the schedulers, in particular the optimal schedulers, induced by the learned models.Comment: In Proceedings QFM 2012, arXiv:1212.345

Should We Learn Probabilistic Models for Model Checking? A New Approach and An Empirical Study

Many automated system analysis techniques (e.g., model checking, model-based testing) rely on first obtaining a model of the system under analysis. System modeling is often done manually, which is often considered as a hindrance to adopt model-based system analysis and development techniques. To overcome this problem, researchers have proposed to automatically "learn" models based on sample system executions and shown that the learned models can be useful sometimes. There are however many questions to be answered. For instance, how much shall we generalize from the observed samples and how fast would learning converge? Or, would the analysis result based on the learned model be more accurate than the estimation we could have obtained by sampling many system executions within the same amount of time? In this work, we investigate existing algorithms for learning probabilistic models for model checking, propose an evolution-based approach for better controlling the degree of generalization and conduct an empirical study in order to answer the questions. One of our findings is that the effectiveness of learning may sometimes be limited.Comment: 15 pages, plus 2 reference pages, accepted by FASE 2017 in ETAP

Data-driven and Model-based Verification: a Bayesian Identification Approach

This work develops a measurement-driven and model-based formal verification approach, applicable to systems with partly unknown dynamics. We provide a principled method, grounded on reachability analysis and on Bayesian inference, to compute the confidence that a physical system driven by external inputs and accessed under noisy measurements, verifies a temporal logic property. A case study is discussed, where we investigate the bounded- and unbounded-time safety of a partly unknown linear time invariant system

Enriched tool support for Probabilistic Specification Mining (ProSpecMi)

Specification Mining describes the process of creating a specification from a (probably unknown) program using sample executions. Most of the current specification miners are deterministic. This thesis aims to create a probabilistic specification miner. Therefor, a specification miner with three different probabililistic approaches has been implemented and added to the LearnLib-Framework. The implementation has been validated by letting the specification miner rebuild a predefined specification to compare the template and the result, by running a hypothesis-test to compare the used approaches to calculate the probabilities against another and by letting it mine the usage of a real API from n tests and validate them with m more tests.Specification Mining beschreibt den Vorgang, eine probabilistische Spezifikation eines (möglicherweise unbekannten) Programms unter Verwendung von Beispielausführungen zu erstellen. Die meisten der aktuellen Specification Miners sind deterministisch. Ziel dieser Ausarbeitung ist ein probabilistischer Specification Miner. Dafür wurde ein Specification Miner mit drei verschiedenen probabilistischen Ansätzen implementiert und dem LearnLib-Framework hinzugefügt. Die Implementierung wurde validiert, indem der Specification Miner eine vorgegebene Spezification nachgebaut hat, um die Vorlage mit dem Ergebnis zu vergleichen, durch einen Hypothesen-Test, der die benutzten Ansätze zur Berechnung der Wahrscheinlichkeiten untereinander verglicht und indem der Specification Miner eine Spezifikation für die Benutzung einer realen API aus n tests erstellt, die mit m weiteren Tests validiert wird

Dynamic Protocol Reverse Engineering a Grammatical Inference Approach

Round trip engineering of software from source code and reverse engineering of software from binary files have both been extensively studied and the state-of-practice have documented tools and techniques. Forward engineering of protocols has also been extensively studied and there are firmly established techniques for generating correct protocols. While observation of protocol behavior for performance testing has been studied and techniques established, reverse engineering of protocol control flow from observations of protocol behavior has not received the same level of attention. State-of-practice in reverse engineering the control flow of computer network protocols is comprised of mostly ad hoc approaches. We examine state-of-practice tools and techniques used in three open source projects: Pidgin, Samba, and rdesktop . We examine techniques proposed by computational learning researchers for grammatical inference. We propose to extend the state-of-art by inferring protocol control flow using grammatical inference inspired techniques to reverse engineer automata representations from captured data flows. We present evidence that grammatical inference is applicable to the problem domain under consideration

Learning Continuous Time Markov Chains from Sample Executions

Continuous-time Markov Chains (CTMCs) are an important class of stochastic models that have been used to model and analyze a variety of practical systems. In this paper we present an algorithm to learn and synthesize a CTMC model from sample executions of a system. Apart from its theoretical interest, we expect our algorithm to be useful in verifying black-box probabilistic systems and in compositionally verifying stochastic components interacting with unknown environments. We have implemented the algorithm and found it to be effective in learning CTMCs underlying practical systems from sample runs