56 research outputs found

    On the connection of probabilistic model checking, planning, and learning for system verification

    Get PDF
    This thesis presents approaches using techniques from the model checking, planning, and learning community to make systems more reliable and perspicuous. First, two heuristic search and dynamic programming algorithms are adapted to be able to check extremal reachability probabilities, expected accumulated rewards, and their bounded versions, on general Markov decision processes (MDPs). Thereby, the problem space originally solvable by these algorithms is enlarged considerably. Correctness and optimality proofs for the adapted algorithms are given, and in a comprehensive case study on established benchmarks it is shown that the implementation, called Modysh, is competitive with state-of-the-art model checkers and even outperforms them on very large state spaces. Second, Deep Statistical Model Checking (DSMC) is introduced, usable for quality assessment and learning pipeline analysis of systems incorporating trained decision-making agents, like neural networks (NNs). The idea of DSMC is to use statistical model checking to assess NNs resolving nondeterminism in systems modeled as MDPs. The versatility of DSMC is exemplified in a number of case studies on Racetrack, an MDP benchmark designed for this purpose, flexibly modeling the autonomous driving challenge. In a comprehensive scalability study it is demonstrated that DSMC is a lightweight technique tackling the complexity of NN analysis in combination with the state space explosion problem.Diese Arbeit präsentiert Ansätze, die Techniken aus dem Model Checking, Planning und Learning Bereich verwenden, um Systeme verlässlicher und klarer verständlich zu machen. Zuerst werden zwei Algorithmen für heuristische Suche und dynamisches Programmieren angepasst, um Extremwerte für Erreichbarkeitswahrscheinlichkeiten, Erwartungswerte für Kosten und beschränkte Varianten davon, auf generellen Markov Entscheidungsprozessen (MDPs) zu untersuchen. Damit wird der Problemraum, der ursprünglich mit diesen Algorithmen gelöst wurde, deutlich erweitert. Korrektheits- und Optimalitätsbeweise für die angepassten Algorithmen werden gegeben und in einer umfassenden Fallstudie wird gezeigt, dass die Implementierung, namens Modysh, konkurrenzfähig mit den modernsten Model Checkern ist und deren Leistung auf sehr großen Zustandsräumen sogar übertrifft. Als Zweites wird Deep Statistical Model Checking (DSMC) für die Qualitätsbewertung und Lernanalyse von Systemen mit integrierten trainierten Entscheidungsgenten, wie z.B. neuronalen Netzen (NN), eingeführt. Die Idee von DSMC ist es, statistisches Model Checking zur Bewertung von NNs zu nutzen, die Nichtdeterminismus in Systemen, die als MDPs modelliert sind, auflösen. Die Vielseitigkeit des Ansatzes wird in mehreren Fallbeispielen auf Racetrack gezeigt, einer MDP Benchmark, die zu diesem Zweck entwickelt wurde und die Herausforderung des autonomen Fahrens flexibel modelliert. In einer umfassenden Skalierbarkeitsstudie wird demonstriert, dass DSMC eine leichtgewichtige Technik ist, die die Komplexität der NN-Analyse in Kombination mit dem State Space Explosion Problem bewältigt

    Modelling safety critical systems with ageing components, with application to underground railway risk and hazards

    Get PDF
    In this thesis methodologies for modelling risk on ageing systems are developed. In the first stages of the thesis, two systems on an underground railway are used to demonstrate the modelling approach. In the latter stages of this thesis the modelling approach is expanded further, presenting a method for optimisation of a phased maintenance strategy, an inclusion of uncertainty in model outputs and an approach to model size reduction. Initially, a Petri net modelling approach is proposed to predict the derailment caused by component failures on a Switch and Crossing (S&C). A holistic methodology is adopted such that components of the system are divided into subsets of interconnected modules at a system level. Degradation within each module is idealized through a sequence of discrete states of wear until final failure occurs. Monte Carlo analysis is used to numerically evaluate the resulting Petri net. Through this methodology, different maintenance strategies, such as partial replacement, complete replacement, and opportunistic maintenance, are tested, to evaluate their influence on the final risk of derailment and predicted system state over time. This work includes a more in-depth modelling approach for S&C than that available in literature. This improves on the state of the art by removing assumptions of perfect maintenance and inspection. In addition, the approach includes modelling of dependencies between components, that are introduced through shared maintenance actions. Secondly, a Petri net modelling approach is applied to an automatic fire protection system to assess the probability of system failure, throughout the system life. Components are modelled with individual Petri nets, which are connected by a phased asset management strategy. The model is solved numerically via Monte Carlo simulation and component failure probabilities are combined using logic developed through Fault Tree analysis. For each time period, this application gives the probability of detection, deluge and alarm system failure, along with the number of maintenance actions, system tests and false system activations. The key contributions from this work include a detailed model for the interlocking fire protection systems and the application of a phased asset management strategy. This phased strategy allows the modelling of different maintenance approaches that are applied at different times depending on the system age. This approach demonstrates an increased functionality in comparison to modelling approaches currently available for fire protection systems, In addition, the modelling approach is extended further towards an optimal risk-based asset management decision making tool. The model for the fire protection systems is used as an application and is extended to give a measure of risk and whole-life cost. This extended model forms the basis of a two-stage optimisation approach within the framework of a phased asset management strategy. A Simulated Annealing algorithm is combined with a Genetic Algorithm to reduce system level risk and whole-life cost. A method for the incorporation of uncertainty in predicted model outputs is also presented. Novel aspects within this work include: the development of the optimisation approach for a phased asset management strategy and the developed algorithm for quantifying model output uncertainty given uncertain input parameters. The optimization of a phased system shows improvements on current model optimisation examples as it allows different strategies to be applied at different phases of the system lifecycle. It allows these phases to be determined in an automatic manner. The inclusion of uncertainty estimates on model outputs improves current Petri net modelling approaches, where uncertainty in input parameters is not included, as it allows decisions based on modelling outcomes to be more fully informed. Finally, a method is presented that can be applied to large system level Petri net models to produce equivalent model at a reduced computational cost. The method consists of generating a reduced Petri net which approximates the behaviour of its larger counterpart with a shorter simulation time. Parameters in this reduced structure are updated following a combined Approximate Bayesian Computation and Subset Simulation framework. Novel contributions from this work include: the proposed reduction approach, a method for using this reduction approach to improve model optimisation efficiency and the exploration of the reduction approach to justify model structure selection. These improve on approaches for model reduction available in literature, which are commonly rule based and so less flexible. In addition, model choice is typically user defined without quantifiable evidence for the suitability of the selected model structure

    Wireless Sensor Networks

    Get PDF
    The aim of this book is to present few important issues of WSNs, from the application, design and technology points of view. The book highlights power efficient design issues related to wireless sensor networks, the existing WSN applications, and discusses the research efforts being undertaken in this field which put the reader in good pace to be able to understand more advanced research and make a contribution in this field for themselves. It is believed that this book serves as a comprehensive reference for graduate and undergraduate senior students who seek to learn latest development in wireless sensor networks

    Modelling methodologies for railway asset management

    Get PDF
    Management of railway assets incurs significant expenditure. Railway asset management modelling can predict the cost and efficacy of an asset management plan, and thus support the asset management planning process. Modelling frameworks can be used to facilitate the development of large, multi-asset, whole life cycle models which can be used to represent large sections of rail track and associated assets. This is achieved with libraries of models and tools with a high level of inter-compatibility. This research set out to support the development of modelling frameworks for railway asset management. It sought to determine the state of the art of railway asset management modelling in order to find which assets require further modelling development before they can be suitably represented in a framework’s model library. It also sought to determine the most accurate and suitable modelling methodology to base the framework upon. These aims were met by first carrying out a literature review to determine the state of the art of asset management modelling for major railway asset types. This review found Petri net models solved via Monte Carlo methods to be the most suitable modelling methodology for asset management. The level crossing asset class was chosen for the development of several models to explore the different types of Petri net model, concentrating on the computational resources required. This asset class was chosen as no asset management model was found in literature, and the diversity of the asset interactions. Literature review found several asset classes in need of further development, and some where asset management modelling may not be possible without other advances. The level crossing Petri net models developed demonstrated that computational requirements differ between the various types of Petri net. Stochastic Petri nets were found to simulate quickly, but had a high memory requirement. Coloured Petri nets were found to have the opposite requirements. A novel Petri net type, the Simple Coloured Petri net was developed to create a balance in computational cost. It was further found that complex processes such as scheduling and resource allocation can only be carried out using Coloured Petri nets due to their enhanced feature set. This work has found that further research on modelling specific asset classes is required to enable the development of a complete asset modelling library for use in a framework. If large models are to be developed, it is recommended that the Simple Coloured Petri net be used to balance computational requirements. Any models requiring complex functions should be developed using the Coloured Petri net methodology

    Towards efficient analysis of Markov automata

    Get PDF
    One of the most expressive formalisms to model concurrent systems is Markov automata. They serve as a semantics for many higher-level formalisms, such as generalised stochastic Petri nets and dynamic fault trees. Two of the most challenging problems for Markov automata to date are (i) the optimal time-bounded reachability probability and (ii) the optimal long-run average rewards. In this thesis, we aim at designing efficient sound techniques to analyse them. We approach the problem of time-bounded reachability from two different angles. First, we study the properties of the optimal solution and exploit this knowledge to construct an efficient algorithm that approximates the optimal values up to a guaranteed error bound. This algorithm is exhaustive, i. e. it computes values for each state of the Markov automaton. This may be a limitation for very large or even infinite Markov automata. To address this issue we design a second algorithm that approximates the optimal solution by only working with part of the total state-space. For the problem of long-run average rewards there exists a polynomial algorithm based on linear programming. Instead of chasing a better theoretical complexity bound we search for a practical solution based on an iterative approach. We design a value iteration algorithm that in our empirical evaluation turns out to scale several orders of magnitude better than the linear programming based approach.Markov-Automaten bilden einen der ausdrucksstärksten Formalismen um Nebenläufige Systeme zu modellieren. Sie werden benutzt um die Semantik vieler höherer Formalismen wie stochastischer Petri-Netze [Mar95, EHZ10] und Dynamic Fault Trees [DBB90] zu beschreiben. Die zwei herausfordernder Probleme im Bereich der Analyse großer Markov- Automaten sind (i) die zeitbeschränkten Erreichbarkeitwahrscheinlichkeit und (ii) optimale langfristige durchschnittliche Rewards. Diese Arbeit zielt auf das Design effizienter und korrekter Techniken um sie zu untersuchen. Das Problem der zeitbeschränkten Erreichbarkeitswahrscheinlichkeit gehen wir aus zwei verschiedenen Richtungen an: Zum einen studieren wir die Eigenschaften optimaler Lösungen und nutzen dieses Wissen um einen effizienten Approximationsalgorithmus zu bilden, der optimale Werte bis auf eine garantierte Fehlertoleranz berechnet. Dieser Algorithmus basiert darauf, Werte für jeden Zustand des Markov-Automaten zu berechnen. Dies kann die Anwendbarkeit für große oder gar unendliche Automaten einschränken. Um diese Problem zu lösen präsentieren wir einen zweiten Algorithmus, der die optimale Lösung approximiert, und dabei ausschließlich einen Teil des Zustandsraumes betrachtet. Für das Problem der optimalen langfristigen durchschnittlichen Rewards gibt es einen polynomiellen Algorithmus auf Basis linearer Programmierung. Anstelle eine bessere theoretische Komplexität anzustreben, konzentrieren wir uns darauf, eine praktische Lösung auf Basis eines iterativen Ansatzes zu finden. Wie entwickeln einen Werte-iterierenden Algorithmus der in unserer empirischen Evaluation um mehrere Größenordnungen besser als der auf linearer Programmierung basierende Ansatz skaliert

    Parameter Synthesis for Markov Models

    Full text link
    Markov chain analysis is a key technique in reliability engineering. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not---or only partially---known. This motivates considering parametric models with transitions labeled with functions over parameters. Whereas traditional Markov chain analysis evaluates a reliability metric for a single, fixed set of probabilities, analysing parametric Markov models focuses on synthesising parameter values that establish a given reliability or performance specification φ\varphi. Examples are: what component failure rates ensure the probability of a system breakdown to be below 0.00000001?, or which failure rates maximise reliability? This paper presents various analysis algorithms for parametric Markov chains and Markov decision processes. We focus on three problems: (a) do all parameter values within a given region satisfy φ\varphi?, (b) which regions satisfy φ\varphi and which ones do not?, and (c) an approximate version of (b) focusing on covering a large fraction of all possible parameter values. We give a detailed account of the various algorithms, present a software tool realising these techniques, and report on an extensive experimental evaluation on benchmarks that span a wide range of applications.Comment: 38 page
    • …
    corecore