APPLICATION AND REFINEMENTS OF THE REPS THEORY FOR SAFETY CRITICAL SOFTWARE

With the replacement of old analog control systems with software-based digital control systems, there is an urgent need for developing a method to quantitatively and accurately assess the reliability of safety critical software systems. This research focuses on proposing a systematic software metric-based reliability prediction method. The method starts with the measurement of a metric. Measurement results are then either directly linked to software defects through inspections and peer reviews or indirectly linked to software defects through empirical software engineering models. Three types of defect characteristics can be obtained, namely, 1) the number of defects remaining, 2) the number and the exact location of the defects found, and 3) the number and the exact location of defects found in an earlier version. Three models, Musa's exponential model, the PIE model and a mixed Musa-PIE model, are then used to link each of the three categories of defect characteristics with reliability respectively. In addition, the use of the PIE model requires mapping defects identified to an Extended Finite State Machine (EFSM) model. A procedure that can assist in the construction of the EFSM model and increase its repeatability is also provided. This metric-based software reliability prediction method is then applied to a safety-critical software used in the nuclear industry using eleven software metrics. Reliability prediction results are compared with the real reliability assessed by using operational failure data. Experiences and lessons learned from the application are discussed. Based on the results and findings, four software metrics are recommended. This dissertation then focuses on one of the four recommended metrics, Test Coverage. A reliability prediction model based on Test Coverage is discussed in detail and this model is further refined to be able to take into consideration more realistic conditions, such as imperfect debugging and the use of multiple testing phases

A Study of Software Input Failure Propagation Mechanisms

Probabilistic Risk Assessment (PRA) is a well-established technique to assess the probability of failure or success of a system. Classical PRA does not consider the contributions of software to risk. Dr. B. Li and C. Smidts have established a framework to integrate software into PRA which recognizes the existence of four classes of risk contributors: functional, input, output and support failures. Input/Output failures have been shown to make up 57.4 % of the failures experienced during software development of major aerospace systems and have been at the origin of a number of major accidents such as the Mars Polar Lander. This research quantifies the contribution of the input failures. More specifically, this dissertation 1) defines the concept of input failure, 2) studies the related propagation mechanisms, 2) estimates the propagation probability for different types of input failures, and 3) applies the fault propagation analysis to the framework of integrating software into PRA. The dissertation defines the concept of artifact as a reference point to identify expected inputs and consequently input failures (inputs which differ from the expected ones). Input failures are divided into value-related failures (including value, range, type and amount failures) and time-related failures (including time, rate and duration failures). Value failures are examined first. The concept of masking areas and flat parts is defined, and the dissertation proposes an Image Reconstruction Method (IRM) to estimate the propagation probability of input value failures. This method is proven to require less number of test cases than one that could be based on random testing to reach the same relative error. For the other input failure modes, the dissertation reveals how they transform to the data state error and formalizes their propagation criteria so that the IRM can be applied to estimate the propagation probability. The contributions are thus: 1. Clear definition of the concept of input failure; 2. Definition of a systematic process of identification and quantification of the contributions of input failures to risk; 3. Systematic analysis of the propagation mechanisms of each type of input failures

Analysis of Errors in Software Reliability Prediction Systems and Application of Model Uncertainty Theory to Provide Better Predictions

Models are the medium by which we reflect and express our understanding of some aspect of reality, a particular unknown of interest. As it is virtually impossible to grasp any situation in its entire complexity, models are representations of reality that are always partial resulting in a state of uncertainty or error. However the question of model error from a pragmatic point of view is not one of accounting for the difference between models and reality at a fundamental level, as such difference always exists. Rather the question is whether the prediction or performance of the model is correct at some practically acceptable level, within the model's domain of application. Here lays the importance of assessing the impact of uncertainties about predictions of a model, modeling the error and trying to reduce the uncertainties associated as much as possible to provide better estimations. While the methods for assessing the impact of errors on the performance of a model and error modeling are well established in various scientific and engineering disciplines, to the best of our knowledge no substantial work has been done in the field of Software Reliability Modeling despite the fact that the inadequacy of the present state and techniques of software reliability estimation has been recognized by industry and government agencies. In summary, even though hundreds of software reliability models have been developed, the software reliability discipline is still struggling to establish a software reliability prediction framework. This work intends to improve the performance of software reliability models through error modeling. It analyzes the errors associated with a set of five software Reliability Prediction Systems (RePSs) and attempts to improve their prediction accuracy using a model uncertainty framework. In the process, this work also statistically validates the performances of the RePSs. It also provides a time and cost effective alternative to performing experiments that are required to assess the error form which is integral to the process of application of the model uncertainty framework

Pfadbedingungen in Abhängigkeitsgraphen und ihre Anwendung in der Softwaresicherheitstechnik

Diese Arbeit präsentiert eine neue Methode zur Sicherheitsanalyse von Software im Bereich der Manipulationsprüfung und der Einhaltung von Informationsflüssen zwischen verschiedenen Sicherheitsniveaus. Program-Slicing und Constraint-Solving sind eigenständige Verfahren, die sowohl zur Abhängigkeitsbestimmung als auch zur Berechnung arithmetischer Eigenschaften verwendet werden. Die erstmalige Kombination dieser beiden Verfahren mittels Pfadbedingungen liefert nicht nur binäre Abhängigkeitsinformationen wie Slicing, sondern exakte notwendige Bedingungen über die Informationsflüsse zwischen zwei Programmpunkten. Neben der Definition der Grundlagen von Abhängigkeitsgraphen und einfachen Pfadbedingungen werden neue Erweiterungen für kontextsensitive interprozedurale Pfadbedingungen gezeigt und die Integration von domänenspezifischen Verfahren für Arrayfelder und abstrakten Datentypen demonstriert. Der Schwerpunkt der Arbeit liegt in der Realisierung von Pfadbedingungen für echte Programme in echten Programmiersprachen. Hierfür werden Verfahren vorgeschlagen, realisiert und empirisch untersucht, wie Pfadbedingungen für große Programme skalieren. Die zum Einsatz kommenden Techniken sind u.a. Intervallanalyse und Binäre Entscheidungsgraphen, mit denen die generelle exponentielle Komplexität von Pfadbedingungen beherrschbar wird. Fallstudien für den Einsatz von Pfadbedingungen und die empirische Untersuchung mehrerer Verfahren zur Intervallanalyse zeigen, dass Pfadbedingungen für die praktische Programmanalyse und das Programmverstehen geeignet und empfehlenswert sind

An Information Flow Model of Fault Detection

Relay is a model of how a fault causes a failure on execution of some test datum. This process begins with introduction of an original state potential failure at a fault location and continues as the potential failure(s) transfers to output. Here we describe the second stage of this process, transfer of an incorrect intermediate state from a faulty statement to output. Transfer occurs along information flow chains, where each link in the chain involves data dependence transfer and/or control dependence transfer. Relay models concurrent transfer along multiple information flow chains with transfer sets, which identify possible interaction between potential failures, and with transfer routes, which identify actual interactions. Transfer sets, transfer routes, and control dependence transfer are unique to the Relay model. The model demonstrates that the process of potential failure transfer is extremely complex and full analysis of real programs may not be practical. Nonetheless, Relay p..

An Information Flow Model of Fault Detection

RELAY is a model of how a fault causes a failure on execution of some test datum. This process begins with Margaret. C. Thompson’ Debra J. Richardson introduction of an original state potential failure at a fault location and continues as the potential failure(s) transfers to output. Here we describe the second stage of this process, transfer of an incorrect intermediate state from a faulty statement to output. Transfer occurs along information flow chains, where each link in the chain involves data dependence trans-fer and/or control dependence transfer. RELAY mod-els concurrent transfer along multiple information flow chains with transfer sets, which identify possible interac-tion bet ween potential failures, and with transfer routes, which identify actual interactions. Transfer sets, trans-fer routes, and control dependence transfer are unique to the RELAY model. The model demonstrates that the process of poten-tial failure transfer is extremely complex and full anal-ysis of real programs may not be practical. Nonethe-less, RELAY provides insight into testing and fault de-tection and suggests an approach to fault-based testing and analysis that may be warranted for critical systems soft ware. This material is based upon work sponsored by the Defense Advanced Research Projects Agency under Grants # h4DA972-91-J-1oo9 and # MDA972-91-J-1012. The content does not nec-essarily reflect the position or the policy of the U.S. Government, and no official endorsement should be inferred. Permission to copy without fee all or part of this material is granted provided that the copies are not made or distributed for direct commercial advantage, the ACM copyright notice and the titla of the publication and its date appear, and notice is givan that copying is by permission of the Association for Computin