Proposing the Use of Hazard Analysis for Machine Learning Data Sets

There is no debating the importance of data for artificial intelligence. The behavior of data-driven machine learning models is determined by the data set, or as the old adage states: “garbage in, garbage out (GIGO).” While the machine learning community is still debating which techniques are necessary and sufficient to assess the adequacy of data sets, they agree some techniques are necessary. In general, most of the techniques being considered focus on evaluating the volumes of attributes. Those attributes are evaluated with respect to anticipated counts of attributes without considering the safety concerns associated with those attributes. This paper explores those techniques to identify instances of too little data and incorrect attributes. Those techniques are important; however, for safety critical applications, the assurance analyst also needs to understand the safety impact of not having specific attributes present in the machine learning data sets. To provide that information, this paper proposes a new technique the authors call data hazard analysis. The data hazard analysis provides an approach to qualitatively analyze the training data set to reduce the risk associated with the GIGO

TBD

I want to tell you a story about an encounter I had at a hotel bar in Lancaster California. I appreciate that at first it doesn’t appear to have anything to do with System Safety. Trust me, I think you will agree that perhaps there is an important lesson for us and the Society

The Difficulties with Replacing Crew Launch Abort Systems with Designed Reliability

As the space industry continues to innovate and new paradigms arise to challenge the status quo, human spaceflight is now perceived as safer and more accessible than ever before. This has led to a new line of thinking in which crewed launch vehicles should be reusable and reliable like commercial airplanes, forgoing the need for an abort system. This paper will counter that line of thought with an analysis of the spectrum of coverage historical crew abort systems provided during launch and use historical data from launch rate successes and failures to glean insight into what reliability in the human spaceflight industry can expect when designing the vehicles of the future. This historical launch vehicle reliability will then be compared to system safety standards used in the commercial aviation industry to understand if future designs truly need a crew abort system. Through this analysis, the rationale for why these crew abort systems have historically been used can be better understood

Review of the Latest Developments in Automotive Safety Standardization for Driving Automation Systems

The ISO 26262: Functional Safety – Road Vehicles Standard has been the de-facto automotive functional safety standard since it was first released in 2011. With the introduction of complex driving automation systems, new standardization efforts to deal with safety of these systems have been initiated to address emerging gaps such as the human/automation roles and responsibilities in the presence/absence of the driver/user, the impact of the technological limitations and the verification and validation needs of automation systems to name a few. This paper highlights some of these gaps and introduces some of the latest developments in automotive safety standardization for driving automation systems

Incremental Assurance Through Eliminative Argumentation

An assurance case for a critical system is valid for that system at a particular point in time, such as when the system is delivered to a certification authority for review. The argument is structured around evidence that exists at that point in time. However, modern assurance cases are rarely one-off exercises. More information might become available (e.g., field data) that could strengthen (or weaken) the validity of the case. This paper proposes the notion of incremental assurance wherein the assurance case structure includes both the currently available evidence and a plan for incrementally increasing confidence in the system as additional or higher quality evidence becomes available. Such evidence is needed to further reduce doubts engineers or reviewers might have. This paper formalizes the idea of incremental assurance through an argumentation pattern. The concept of incremental assurance is demonstrated by applying the pattern to part of a safety assurance case for an air traffic control system

System Safety Bookshelf: System Safety for the 21st Century, 2nd Edition

Over many decades System Safety has evolved from a more re-active nature - learning from failures and improving – not really suitable for high consequence enterprises - to today’s more pro-active form. This is now based on better fundamental understanding, better assessment processes, better standards, more comprehensive analysis tools with better audit and regulation procedures. However, unlike ‘set educational subjects’ such as engineering, science, technology and mathematics, there are less opportunities for formal System Safety education and training in academia and elsewhere, even though system safety impacts on all aspects of life. One hopes that this will continue to be rectified. This leads us directly to the importance and value of this book, which gives a complete insight into the nature of what System Safety is all about, including its approaches, methodologies and tools, and which provides guidance on the successful application of a comprehensive, pro-active approach for ensuring safe system design

Human Reliability Analysis using a Human Factors Hazard Model

Human Reliability Analysis (HRA) has found application within a diverse set of engineering domains, but the methods used to apply HRA are often complicated, time-consuming, costly to apply, specific to particular (i.e., nuclear) applications, and are not suitable for direct comparison amongst themselves. This paper proposes a Human Factors Hazard Model (HFHM), which builds an HRA method from the tools of Fault Tree Analysis (FTA), Event Tree Analysis (ETA), and a novel model of considering serial Human Error Probability (HEP) more relevant to psychomotor-intensive industrial and commercial applications such as manufacturing, teleoperation, and vehicle operation. The HEP approach uses Performance Shaping Factors (PSFs) relevant to human behavior, as well as specific characteristics unique to a system architecture and its corresponding operational behavior. The HFHM tool is intended to establish a common analysis approach, to simplify and automate the modeling of the likelihood of a mishap due to a human-system interaction during a hazard event. The HFHM is executed commercial software tools (MS Excel and SysML) such that trade and sensitivity studies can be conducted and iterated automatically. The results generated by the HFHM can be used to guide risk assessment, safety requirements generation and management, design options, and safety controls within the system design architecting process. Verification and evaluation of the HFHM through simulation and subject matter expert evaluation illustrate the value of the HFHM as a tool for HRA and system safety analysis in a set of key industrial applications

From Our Readers

Letter to the Editor: In the Charles Hoes article “TBD regarding Risk Assessment” the sample risk assessment matrix and his explanation of how the chart can be used to assign “risk levels” is on par with the basics of the risk management process generally used on programs

From the Editor's Desk: Change

At the last Executive Council meeting of the International System Safety Society (ISSS), a decision was made to temporarily reduce the number of Journal of System Safety issues to three hard copies per year. This measure was necessary in order to reduce expenses that are currently exceeding income. We’ll continue to provide a high-quality journal by including a little more material in each issue to help make up for the loss. If more members would pay their dues and encourage non-members to join the ISSS, our income would increase enough to go back to the normal number of issues

Quantification of Benefits for Medical Devices

One of the most prominent challenges in safety risk management of medical devices is the Benefit-Risk Analysis. This paper proposes a methodology to quantify benefits, thereby creating more consistency, and explainability in the evaluation of benefits and the benefit/risk ratio. Leveraging the guidance from the FDA, we define four Dimensions for appraising benefits. The product of the rankings of a benefit in all four Dimensions is used as a quantitative measure of a benefit. The quantitative score for the overall benefit of a medical device would be the sum of the scores of the individual benefits