Baseline Assessment and Prioritization Framework for IVHM Integrity Assurance Enabling Capabilities

Fundamental to vehicle health management is the deployment of systems incorporating advanced technologies for predicting and detecting anomalous conditions in highly complex and integrated environments. Integrated structural integrity health monitoring, statistical algorithms for detection, estimation, prediction, and fusion, and diagnosis supporting adaptive control are examples of advanced technologies that present considerable verification and validation challenges. These systems necessitate interactions between physical and software-based systems that are highly networked with sensing and actuation subsystems, and incorporate technologies that are, in many respects, different from those employed in civil aviation today. A formidable barrier to deploying these advanced technologies in civil aviation is the lack of enabling verification and validation tools, methods, and technologies. The development of new verification and validation capabilities will not only enable the fielding of advanced vehicle health management systems, but will also provide new assurance capabilities for verification and validation of current generation aviation software which has been implicated in anomalous in-flight behavior. This paper describes the research focused on enabling capabilities for verification and validation underway within NASA s Integrated Vehicle Health Management project, discusses the state of the art of these capabilities, and includes a framework for prioritizing activities

Got Controversy - Milk Does

This article analyzes ongoing controversy over how to best label rBST-free milk. Recombinant bovine somatotropin is a genetically engineered drug administered by some farmers to their dairy herds to increase milk production. FDA first approved its use in 1994, despite great controversy. The FDA also issued labeling guidelines that allowed voluntary disclosure of rBST-free milk, so long as it carried the disclaimer that no difference could be detected between milk produced with rBST and rBST-free. The controversy continues today as consumers express a preference for rBST-free milk and many rBST-free producers label their milk this way. Conventional milk (with rBST) is viewed by the FDA as materially the same as rBST-free. So, conventional producers have continually challenged marketing that is meant to convey rBST free milk is superior or more nutritious. This article specifically analyzes the recent activities surrounding rBST - an FTC petition, an FDA petition to withdraw rBST approval, and several states\u27 rulemaking and legislation to tighten rBST-free labeling. One of the state\u27s rules has also spawned federal litigation. The article recommends that stakeholders must generate consumer survey data to understand the effect of rBST labels on consumers, and then use that data to design better milk labels

A Harmonized Compositional Assurance Approach for Safety-Critical Systems

Safety-critical systems, those whose failure could end up in loss or injuries to people or the environment, are required to go through laborious and expensive certification processes. These systems have also increased their complexity and as it has already been done in other domains, they have applied component-based system developments to deal with complexity. However, components are difficult to assess as certification is done at system level and not at component level. Compositional certification approach proposes to get incremental credit by accepting that a specific component complies with specific standard’s requirements and it is correctly integrated. The objective is to support integration of new components while the previously integrated components do not need to work for re-acceptance. We propose (1) the use of assurance modelling techniques to provide us the mechanism to understand the common basis of standards shared by different domains such as the avionics, automotive and the medical devices design. We propose (2) an assurance decomposition methodology offering guidance and modelling mechanisms to decompose the responsibilities associated with the life-cycle of safety-critical components. This methodology ensures a hierarchy of assurance and certification projects where the responsibilities and project tasks can be specified and its accomplishment can be assessed to determine the compliance of functional safety standards. Assurance decomposition supports the reuse of components as it guides us not just for standards compliance but specifically on the understanding and tailoring of those standards for component assurance and support when those components are integrated into the final system. We propose (3) a contract-based approach to support the integration of reused components and at the same time, the proposal supports the identification of assumptions, a very laborious and time consuming task. Assurance Contracts are defined to ensure incremental compliance once the components are integrated. The objective of this assurance contracts is to ensure the overall compliance of the system with the selected standards and reference documents such as guidelines or advisory circulars. The defined approach to assurance contracts specification attempts to balance the need for unambiguity on the composition while maintaining the heterogeneity of the information managed. The claims classification offers an easy method to support the assessment of contract completeness and the structured expressions provide a semi-formal language to specify the assumptions and guarantees of a component. This work has been mainly framed in a European collaborative research projects such as OPENCOSS a Large-scale integrating project (IP) with 17 partners from 9 countries to develop a platform for safety assurance and certification of safety-critical systems (compliance with standards, robust argumentation, evidence management, process transparency), SAFEADAPT an FP7 project with 9 partners and RECOMP an ARTEMIS project.. The results of this work have been presented to the standardization group of the Object Management Group responsible for the SACM (Structured Assurance Case Metamodel) standard specification, which currently discusses its inclusion in future versions. The (4) tools presented and used in this work have been included in the results of an open tool platform developed within the OPENCOSS project that is being released in PolarSys. PolarSys is an Eclipse Industry Working Group created by large industry players and by tools providers to collaborate on the creation and support of Open Source tools for the development of embedded systems

Proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification

This NASA conference publication contains the proceedings of the Third International Workshop on Proof-Carrying Code and Software Certification, held as part of LICS in Los Angeles, CA, USA, on August 15, 2009. Software certification demonstrates the reliability, safety, or security of software systems in such a way that it can be checked by an independent authority with minimal trust in the techniques and tools used in the certification process itself. It can build on existing validation and verification (V&V) techniques but introduces the notion of explicit software certificates, Vvilich contain all the information necessary for an independent assessment of the demonstrated properties. One such example is proof-carrying code (PCC) which is an important and distinctive approach to enhancing trust in programs. It provides a practical framework for independent assurance of program behavior; especially where source code is not available, or the code author and user are unknown to each other. The workshop wiII address theoretical foundations of logic-based software certification as well as practical examples and work on alternative application domains. Here "certificate" is construed broadly, to include not just mathematical derivations and proofs but also safety and assurance cases, or any fonnal evidence that supports the semantic analysis of programs: that is, evidence about an intrinsic property of code and its behaviour that can be independently checked by any user, intermediary, or third party. These guarantees mean that software certificates raise trust in the code itself, distinct from and complementary to any existing trust in the creator of the code, the process used to produce it, or its distributor. In addition to the contributed talks, the workshop featured two invited talks, by Kelly Hayhurst and Andrew Appel. The PCC 2009 website can be found at http://ti.arc.nasa.gov /event/pcc 091

Deriving safety cases for hierarchical structure in model-based development

Model-based development and automated code generation are increasingly used for actual production code, in particular in mathematical and engineering domains. However, since code generators are typically not qualified, there is no guarantee that their output satisfies the system requirements, or is even safe. Here we present an approach to systematically derive safety cases that argue along the hierarchical structure in model-based development. The safety cases are constructed mechanically using a formal analysis, based on automated theorem proving, of the automatically generated code. The analysis recovers the model structure and component hierarchy from the code, providing independent assurance of both code and model. It identifies how the given system safety requirements are broken down into component requirements, and where they are ultimately established, thus establishing a hierarchy of requirements that is aligned with the hierarchical model structure. The derived safety cases reflect the results of the analysis, and provide a high-level argument that traces the requirements on the model via the inferred model structure to the code. We illustrate our approach on flight code generated from hierarchical Simulink models by Real-Time Worksho

Robust and Accurate -- Compositional Architectures for Randomized Smoothing

Randomized Smoothing (RS) is considered the state-of-the-art approach to obtain certifiably robust models for challenging tasks. However, current RS approaches drastically decrease standard accuracy on unperturbed data, severely limiting their real-world utility. To address this limitation, we propose a compositional architecture, ACES, which certifiably decides on a per-sample basis whether to use a smoothed model yielding predictions with guarantees or a more accurate standard model without guarantees. This, in contrast to prior approaches, enables both high standard accuracies and significant provable robustness. On challenging tasks such as ImageNet, we obtain, e.g., $80.0\%$ natural accuracy and $28.2\%$ certifiable accuracy against $\ell_2$ perturbations with $r=1.0$. We release our code and models at https://github.com/eth-sri/aces.Comment: Presented at the ICLR 2022 Workshop on Socially Responsible Machine Learnin

Underapproximation of Procedure Summaries for Integer Programs

We show how to underapproximate the procedure summaries of recursive programs over the integers using off-the-shelf analyzers for non-recursive programs. The novelty of our approach is that the non-recursive program we compute may capture unboundedly many behaviors of the original recursive program for which stack usage cannot be bounded. Moreover, we identify a class of recursive programs on which our method terminates and returns the precise summary relations without underapproximation. Doing so, we generalize a similar result for non-recursive programs to the recursive case. Finally, we present experimental results of an implementation of our method applied on a number of examples.Comment: 35 pages, 3 figures (this report supersedes the STTT version which in turn supersedes the TACAS'13 version

New Directions in Robust Time-Series Machine Learning Theory, Algorithms, and Applications

Despite the rapid progress in research on the robustness of deep neural networks (DNNs) for images and text, there is little principled work for the time-series domain. Since time-series data arises in diverse applications, including mobile health, finance, and smart grid, it is important to verify and improve the robustness of DNNs for the time-series domain. Safe deployment of time-series DNNs for real-world applications relies on their ability to be resilient against natural/adversarial perturbations and anomalous inputs that may affect their predictive performance. This dissertation studies the design of robust machine learning (ML) algorithms that aim to minimize both the risk and uncertainty of wrongful decisions made by time-series-based ML systems from both theoretical and algorithmic perspectives.First, we investigate the robustness against adversarial time-series inputs. Adversarial examples were shown to be successful in exposing fundamental blind spots in ML models. While adversarial examples expose how to break the models, the process of creating adversarialexamples can itself improve the robustness of ML models by adding them to the training set. The time-series modality poses unique challenges for studying adversarial robustness that are not seen in images and text. The key challenge is how to assess the similarity in the time-series input space to efficiently create valid time-series adversarial examples. Second, we investigate the challenge of Out-of-Distribution (OOD) detection, where the ML system is required to identify time-series inputs that do not follow the distribution of training data. This is a critical task as deep models often make predictions that are very confident yet incorrect on such examples. Detecting OOD examples is challenging, and the potential risks are high for sensitive applications. The key challenge for time-series inputs is how to identify the features that improve the separability between OOD examples and training examples.Motivated by these goals, this dissertation proposes and evaluates a suite of novel solutions to push the frontiers of robust time-series ML: 1) The practical threats of adversarial examples to time-series ML systems; 2) The use of constraints on statistical features of thetime-series data to construct adversarial examples, and providing formal robustness certificates for time-series data; 3) The use of elastic measures such as Dynamic Time Warping to quantify the similarity between time-series examples and developing theoretically-soundalgorithms to efficiently construct valid adversarial examples, and to train robust ML models by explicitly solving a min-max optimization problem; 4) Adapting and applying the developed algorithms to real-world applications including wearable sensors enabled ML systemsfor healthcare to handle both natural perturbations and missing sensor data; and 5) A novel OOD detection algorithm based on deep generative models for the time-series domain and explain why prior OOD methods from the other domains perform poorly

Model-connected safety cases

Regulatory authorities require justification that safety-critical systems exhibit acceptable levels of safety. Safety cases are traditionally documents which allow the exchange of information between stakeholders and communicate the rationale of how safety is achieved via a clear, convincing and comprehensive argument and its supporting evidence. In the automotive and aviation industries, safety cases have a critical role in the certification process and their maintenance is required throughout a system’s lifecycle. Safety-case-based certification is typically handled manually and the increase in scale and complexity of modern systems renders it impractical and error prone.Several contemporary safety standards have adopted a safety-related framework that revolves around a concept of generic safety requirements, known as Safety Integrity Levels (SILs). Following these guidelines, safety can be justified through satisfaction of SILs. Careful examination of these standards suggests that despite the noticeable differences, there are converging aspects. This thesis elicits the common elements found in safety standards and defines a pattern for the development of safety cases for cross-sector application. It also establishes a metamodel that connects parts of the safety case with the target system architecture and model-based safety analysis methods. This enables the semi- automatic construction and maintenance of safety arguments that help mitigate problems related to manual approaches. Specifically, the proposed metamodel incorporates system modelling, failure information, model-based safety analysis and optimisation techniques to allocate requirements in the form of SILs. The system architecture and the allocated requirements along with a user-defined safety argument pattern, which describes the target argument structure, enable the instantiation algorithm to automatically generate the corresponding safety argument. The idea behind model-connected safety cases stemmed from a critical literature review on safety standards and practices related to safety cases. The thesis presents the method, and implemented framework, in detail and showcases the different phases and outcomes via a simple example. It then applies the method on a case study based on the Boeing 787’s brake system and evaluates the resulting argument against certain criteria, such as scalability. Finally, contributions compared to traditional approaches are laid out