5 research outputs found
Proposing the Use of Hazard Analysis for Machine Learning Data Sets
There is no debating the importance of data for artificial intelligence. The behavior of data-driven machine learning models is determined by the data set, or as the old adage states: “garbage in, garbage out (GIGO).” While the machine learning community is still debating which techniques are necessary and sufficient to assess the adequacy of data sets, they agree some techniques are necessary. In general, most of the techniques being considered focus on evaluating the volumes of attributes. Those attributes are evaluated with respect to anticipated counts of attributes without considering the safety concerns associated with those attributes. This paper explores those techniques to identify instances of too little data and incorrect attributes. Those techniques are important; however, for safety critical applications, the assurance analyst also needs to understand the safety impact of not having specific attributes present in the machine learning data sets. To provide that information, this paper proposes a new technique the authors call data hazard analysis. The data hazard analysis provides an approach to qualitatively analyze the training data set to reduce the risk associated with the GIGO
Testing and verification of neural-network-based safety-critical control software: A systematic literature review
Context: Neural Network (NN) algorithms have been successfully adopted in a
number of Safety-Critical Cyber-Physical Systems (SCCPSs). Testing and
Verification (T&V) of NN-based control software in safety-critical domains are
gaining interest and attention from both software engineering and safety
engineering researchers and practitioners. Objective: With the increase in
studies on the T&V of NN-based control software in safety-critical domains, it
is important to systematically review the state-of-the-art T&V methodologies,
to classify approaches and tools that are invented, and to identify challenges
and gaps for future studies. Method: We retrieved 950 papers on the T&V of
NN-based Safety-Critical Control Software (SCCS). To reach our result, we
filtered 83 primary papers published between 2001 and 2018, applied the
thematic analysis approach for analyzing the data extracted from the selected
papers, presented the classification of approaches, and identified challenges.
Conclusion: The approaches were categorized into five high-order themes:
assuring robustness of NNs, assuring safety properties of NN-based control
software, improving the failure resilience of NNs, measuring and ensuring test
completeness, and improving the interpretability of NNs. From the industry
perspective, improving the interpretability of NNs is a crucial need in
safety-critical applications. We also investigated nine safety integrity
properties within four major safety lifecycle phases to investigate the
achievement level of T&V goals in IEC 61508-3. Results show that correctness,
completeness, freedom from intrinsic faults, and fault tolerance have drawn
most attention from the research community. However, little effort has been
invested in achieving repeatability; no reviewed study focused on precisely
defined testing configuration or on defense against common cause failure.Comment: This paper had been submitted to Journal of Information and Software
Technology on April 20, 2019,Revised 5 December 2019, Accepted 6 March 2020,
Available online 7 March 202