Quantifying Assurance in Learning-enabled Systems

Dependability assurance of systems embedding machine learning(ML) components---so called learning-enabled systems (LESs)---is a key step for their use in safety-critical applications. In emerging standardization and guidance efforts, there is a growing consensus in the value of using assurance cases for that purpose. This paper develops a quantitative notion of assurance that an LES is dependable, as a core component of its assurance case, also extending our prior work that applied to ML components. Specifically, we characterize LES assurance in the form of assurance measures: a probabilistic quantification of confidence that an LES possesses system-level properties associated with functional capabilities and dependability attributes. We illustrate the utility of assurance measures by application to a real world autonomous aviation system, also describing their role both in i) guiding high-level, runtime risk mitigation decisions and ii) as a core component of the associated dynamic assurance case.Comment: Author's pre-print version of manuscript accepted for publication in the Proceedings of the 39th International Conference in Computer Safety, Reliability, and Security (SAFECOMP 2020

Towards Quantification of Assurance for Learning-enabled Components

Perception, localization, planning, and control, high-level functions often organized in a so-called pipeline, are amongst the core building blocks of modern autonomous (ground, air, and underwater) vehicle architectures. These functions are increasingly being implemented using learning-enabled components (LECs), i.e., (software) components leveraging knowledge acquisition and learning processes such as deep learning. Providing quantified component-level assurance as part of a wider (dynamic) assurance case can be useful in supporting both pre-operational approval of LECs (e.g., by regulators), and runtime hazard mitigation, e.g., using assurance-based failover configurations. This paper develops a notion of assurance for LECs based on i) identifying the relevant dependability attributes, and ii) quantifying those attributes and the associated uncertainty, using probabilistic techniques. We give a practical grounding for our work using an example from the aviation domain: an autonomous taxiing capability for an unmanned aircraft system (UAS), focusing on the application of LECs as sensors in the perception function. We identify the applicable quantitative measures of assurance, and characterize the associated uncertainty using a non-parametric Bayesian approach, namely Gaussian process regression. We additionally discuss the relevance and contribution of LEC assurance to system-level assurance, the generalizability of our approach, and the associated challenges.Comment: 8 pp, 4 figures, Appears in the proceedings of EDCC 201

Flow stress identification of tubular materials using the progressive inverse identification method

PURPOSE : Propose a progressive inverse identification algorithm to characterize flow stress of tubular materials from the material response, independent of choosing an a priori hardening constitutive model. DESIGN /METHODOLOGY / APPROACH : In contrast to the conventional forward flow stress identification methods, the flow stress is characterized by a multi-linear curve rather than a limited number of hardening model parameters. The proposed algorithm optimizes the slopes and lengths of the curve increments simultaneously. The objective of the optimization is that the finite element simulation response of the test estimates the material response within a predefined accuracy. FINDINGS : We employ the algorithm to identify flow stress of a 304 stainless steel tube in a tube bulge test as an example to illustrate application of the algorithm. Comparing response of the finite element simulation using the obtained flow stress with the material response shows that the method can accurately determine the flow stress of the tube. PRACTICAL IMPLICATIONS : The obtained flow stress can be employed for more accurate finite element simulation of the metal forming processes as the material behaviour can be characterized in a similar state of stress as the target metal forming process. Moreover, since there is no need for a priori choosing the hardening model, there is no risk for choosing an improper hardening model, which in turn facilitates solving the inverse problem. ORIGINALITY / VALUE : The proposed algorithm is more efficient than the conventional inverse flow stress identification methods. In the latter, each attempt to select a more accurate hardening model, if it is available, result in constructing an entirely new inverse problem. However, this problem is avoided in the proposed algorithm.http://www.emeraldinsight.com/loi/echb2016Mechanical and Aeronautical Engineerin

The use of direct inverse maps to solve material identification problems : pitfalls and solutions

Material parameter identification is a technique that is used to calibrate material models, often a precursor to perform an industrial analysis. Conventional material parameter identification methods estimate the material parameters for a material model by solving an optimisation problem. An alternative but lesser-known approach, called a direct inverse map, directly maps the measured response to the parameters of a material model. In this study we investigate the potential pitfalls of the well-known stochastic noise and lesser-known model errors when constructing direct inverse maps. We show how to address these problems, explaining in particular the importance of projecting the measured response onto the domain of the simulated responses before mapping it to the material parameters. This paper concludes by proposing partial least squares regression as an elegant and computationally efficient approach to address stochastic and systematic (model) errors. This paper also gives insight into the nature of the inverse problem under consideration.National Research Foundation (NRF), the Technology and Human Resources for Industry Programme (THRIP), and the Eskom Power Plant Engineering Institute (EPPEI).http://link.springer.com/journal/1582018-01-31hb2016Mechanical and Aeronautical Engineerin

On the value of test data for reducing uncertainty in material models : computational framework and application to spherical indentation

We present a conceptual framework and the computational tools to study the value of the material responses in designing material characterization tests to identify the material model under uncertainty. A computational framework is first developed to estimate the information gained by observing a material response as a measure of the value of the experiment. The proposed framework is then extended to estimate the mutual information between the material response space and the material model space as a basis for ranking the available material response candidates as they relate to reducing the uncertainty of the inferred model. We then define a design problem where a tunable parameter, referred to as the design parameter, is identified so as to render two different material responses to be of the same value from an information content point of view. We finally study the value of the material responses, obtained in a spherical indentation test, i.e. reaction force–indenter displacement, maximum indentation load and the residual imprint, where it is shown that the proposed framework offers a computationally affordable and uncertainty-aware platform to design material characterization tests.Partially supported by the National Science Foundation (NSF), United States , Grants CMMI-1235238 and CMMI-1351742. The first and second authors would also like to acknowledge UP postdoctoral fellowship program and the Eskom Power Plant Engineering Institute (EPPEI).http://www.elsevier.com/locate/cma2020-04-01hj2019Mechanical and Aeronautical Engineerin

A promising azeotrope-like mosquito repellent blend

Abstract Topical repellents play a key role in reducing the outdoor transmission of mosquito-borne diseases by reducing human-vector contact. Excellent repellents are available, but there is always room for improvement. This article reports on a particularly effective binary repellent blend of ethyl butylacetylaminopropionate and nonanoic acid. A composition containing 25 mol% of the acid exhibits negative pseudo-azeotrope behaviour at 50 °C, meaning that the liquid vapour pressure is lower than that of the parent compounds and evaporation occurs without a change in the liquid composition. In tests performed using the South African Medical Research Council’s cup-on-arm procedure, this mixture provided better protection for a longer time than the “gold standard of mosquito repellents”, namely N,N-diethyl-m-toluamide, commonly known as DEET