Evidence Arguments for Using Formal Methods in Software Certification

We describe a generic approach for automatically integrating the output generated from a formal method/tool into a software safety assurance case, as an evidence argument, by (a) encoding the underlying reasoning as a safety case pattern, and (b) instantiating it using the data produced from the method/tool. We believe this approach not only improves the trustworthiness of the evidence generated from a formal method/tool, by explicitly presenting the reasoning and mechanisms underlying its genesis, but also provides a way to gauge the suitability of the evidence in the context of the wider assurance case. We illustrate our work by application to a real example-an unmanned aircraft system- where we invoke a formal code analysis tool from its autopilot software safety case, automatically transform the verification output into an evidence argument, and then integrate it into the former

Proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification

This NASA conference publication contains the proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification, held as part of LICS in Los Angeles, CA, USA, on August 15, 2009. Software certification demonstrates the reliability, safety, or security of software systems in such a way that it can be checked by an independent authority with minimal trust in the techniques and tools used in the certification process itself. It can build on existing validation and verification (V&V) techniques but introduces the notion of explicit software certificates, Vvilich contain all the information necessary for an independent assessment of the demonstrated properties. One such example is proof-carrying code (PCC) which is an important and distinctive approach to enhancing trust in programs. It provides a practical framework for independent assurance of program behavior; especially where source code is not available, or the code author and user are unknown to each other. The workshop wiII address theoretical foundations of logic-based software certification as well as practical examples and work on alternative application domains. Here "certificate" is construed broadly, to include not just mathematical derivations and proofs but also safety and assurance cases, or any fonnal evidence that supports the semantic analysis of programs: that is, evidence about an intrinsic property of code and its behaviour that can be independently checked by any user, intermediary, or third party. These guarantees mean that software certificates raise trust in the code itself, distinct from and complementary to any existing trust in the creator of the code, the process used to produce it, or its distributor. In addition to the contributed talks, the workshop featured two invited talks, by Kelly Hayhurst and Andrew Appel. The PCC 2009 website can be found at http://ti.arc.nasa.gov /event/pcc 091

Safety Case Patterns: Theory and Applications

We develop the foundations for a theory of patterns of safety case argument structures, clarifying the concepts involved in pattern specification, including choices, labeling, and well-founded recursion. We specify six new patterns in addition to those existing in the literature. We give a generic way to specify the data required to instantiate patterns and a generic algorithm for their instantiation. This generalizes earlier work on generating argument fragments from requirements tables. We describe an implementation of these concepts in AdvoCATE, the Assurance Case Automation Toolset, showing how patterns are defined and can be instantiated. In particular, we describe how our extended notion of patterns can be specified, how they can be instantiated in an interactive manner, and, finally, how they can be automatically instantiated using our algorithm

A Semantic Basis for Proof Queries and Transformations

We extend the query language PrQL, designed for inspecting machine representations of proofs, to also allow transformation of proofs. PrQL natively supports hiproofs which express proof structure using hierarchically nested labelled trees, which we claim is a natural way of taming the complexity of huge proofs. Query-driven transformations enable manipulation of this structure, in particular, to transform proofs produced by interactive theorem provers into forms that assist their understanding, or that could be consumed by other tools. In this paper we motivate and define basic transformation operations, using an abstract denotational semantics of hiproofs and queries. This extends our previous semantics for queries based on syntactic tree representations.We define update operations that add and remove sub-proofs, and manipulate the hierarchy to group and ungroup nodes. We show tha

AdvoCATE - User Guide

The basic vision of AdvoCATE is to automate the creation, manipulation, and management of large-scale assurance cases based on a formal theory of argument structures. Its main purposes are for creating and manipulating argument structures for safety assurance cases using the Goal Structuring Notation (GSN), and as a test bed and proof-of-concept for the formal theory of argument structures. AdvoCATE is available for Windows 7, Macintosh OSX, and Linux. Eventually, AdvoCATE will serve as a dashboard for safety related information and provide an infrastructure for safety decisions and management

Hierarchical Safety Cases

We introduce hierarchical safety cases (or hicases) as a technique to overcome some of the difficulties that arise creating and maintaining industrial-size safety cases. Our approach extends the existing Goal Structuring Notation with abstraction structures, which allow the safety case to be viewed at different levels of detail. We motivate hicases and give a mathematical account of them as well as an intuition, relating them to other related concepts. We give a second definition which corresponds closely to our implementation of hicases in the AdvoCATE Assurance Case Editor and prove the correspondence between the two. Finally, we suggest areas of future enhancement, both theoretically and practically

Querying Safety Cases

Querying a safety case to show how the various stakeholders' concerns about system safety are addressed has been put forth as one of the benefits of argument-based assurance (in a recent study by the Health Foundation, UK, which reviewed the use of safety cases in safety-critical industries). However, neither the literature nor current practice offer much guidance on querying mechanisms appropriate for, or available within, a safety case paradigm. This paper presents a preliminary approach that uses a formal basis for querying safety cases, specifically Goal Structuring Notation (GSN) argument structures. Our approach semantically enriches GSN arguments with domain-specific metadata that the query language leverages, along with its inherent structure, to produce views. We have implemented the approach in our toolset AdvoCATE, and illustrate it by application to a fragment of the safety argument for an Unmanned Aircraft System (UAS) being developed at NASA Ames. We also discuss the potential practical utility of our query mechanism within the context of the existing framework for UAS safety assurance

An Autonomous Autopilot Control System Design for Small-Scale UAVs

This paper describes the design and implementation of a fully autonomous and programmable autopilot system for small scale autonomous unmanned aerial vehicle (UAV) aircraft. This system was implemented in Reflection and has flown on the Exploration Aerial Vehicle (EAV) platform at NASA Ames Research Center, currently only as a safety backup for an experimental autopilot. The EAV and ground station are built on a component-based architecture called the Reflection Architecture. The Reflection Architecture is a prototype for a real-time embedded plug-and-play avionics system architecture which provides a transport layer for real-time communications between hardware and software components, allowing each component to focus solely on its implementation. The autopilot module described here, although developed in Reflection, contains no design elements dependent on this architecture

Automating the Generation of Heterogeneous Aviation Safety Cases

A safety case is a structured argument, supported by a body of evidence, which provides a convincing and valid justification that a system is acceptably safe for a given application in a given operating environment. This report describes the development of a fragment of a preliminary safety case for the Swift Unmanned Aircraft System. The construction of the safety case fragment consists of two parts: a manually constructed system-level case, and an automatically constructed lower-level case, generated from formal proof of safety-relevant correctness properties. We provide a detailed discussion of the safety considerations for the target system, emphasizing the heterogeneity of sources of safety-relevant information, and use a hazard analysis to derive safety requirements, including formal requirements. We evaluate the safety case using three classes of metrics for measuring degrees of coverage, automation, and understandability. We then present our preliminary conclusions and make suggestions for future work

Assuring Ground-Based Detect and Avoid for UAS Operations

One of the goals of the Marginal Ice Zones Observations and Processes Experiment (MIZOPEX) NASA Earth science mission was to show the operational capabilities of Unmanned Aircraft Systems (UAS) when deployed on challenging missions, in difficult environments. Given the extreme conditions of the Arctic environment where MIZOPEX measurements were required, the mission opted to use a radar to provide a ground-based detect-and-avoid (GBDAA) capability as an alternate means of compliance (AMOC) with the see-and-avoid federal aviation regulation. This paper describes how GBDAA safety assurance was provided by interpreting and applying the guidelines in the national policy for UAS operational approval. In particular, we describe how we formulated the appropriate safety goals, defined the processes and procedures for system safety, identified and assembled the relevant safety verification evidence, and created an operational safety case in compliance with Federal Aviation Administration (FAA) requirements. To the best of our knowledge, the safety case, which was ultimately approved by the FAA, is the first successful example of non-military UAS operations using GBDAA in the U.S. National Airspace System (NAS), and, therefore, the first nonmilitary application of the safety case concept in this context