A controlled experiment for the empirical evaluation of safety analysis techniques for safety-critical software

Context: Today's safety critical systems are increasingly reliant on software. Software becomes responsible for most of the critical functions of systems. Many different safety analysis techniques have been developed to identify hazards of systems. FTA and FMEA are most commonly used by safety analysts. Recently, STPA has been proposed with the goal to better cope with complex systems including software. Objective: This research aimed at comparing quantitatively these three safety analysis techniques with regard to their effectiveness, applicability, understandability, ease of use and efficiency in identifying software safety requirements at the system level. Method: We conducted a controlled experiment with 21 master and bachelor students applying these three techniques to three safety-critical systems: train door control, anti-lock braking and traffic collision and avoidance. Results: The results showed that there is no statistically significant difference between these techniques in terms of applicability, understandability and ease of use, but a significant difference in terms of effectiveness and efficiency is obtained. Conclusion: We conclude that STPA seems to be an effective method to identify software safety requirements at the system level. In particular, STPA addresses more different software safety requirements than the traditional techniques FTA and FMEA, but STPA needs more time to carry out by safety analysts with little or no prior experience.Comment: 10 pages, 1 figure in Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering (EASE '15). ACM, 201

Evaluating and Improving Risk Analysis Methods for Critical Systems

At the same time as our dependence on IT systems increases, the number of reports of problems caused by failures of critical IT systems has also increased. Today, almost every societal system or service, e.g., water supply, power supply, transportation, depends on IT systems, and failures of these systems have serious and negative effects on society. In general, public organizations are responsible for delivering these services to society. Risk analysis is an important activity for the development and operation of critical IT systems, but the increased complexity and size of critical systems put additional requirements on the effectiveness of risk analysis methods. Even if a number of methods for risk analysis of technical systems exist, the failure behavior of information systems is typically very different from mechanical systems. Therefore, risk analysis of IT systems requires different risk analysis techniques, or at least adaptations of traditional approaches. The research objective of this thesis is to improve the analysis process of risks pertaining to critical IT systems, which is addressed in the following three ways. First, by understanding current literature and practices related to risk analysis of IT systems, then by evaluating and comparing existing risk analysis methods, and by suggesting improvements in the risk analysis process and by developing new effective and efficient risk analysis methods to analyze IT systems. To understand current risk analysis methods and practices we carried out a systematic mapping study. The study found only few empirical research papers on the evaluation of existing risk analysis methods. The results of the study suggest to empirically investigate risk analysis methods for analyzing IT systems to conclude which methods are more effective than others. Then, we carried out a semi-structured interview study to investigate several factors regarding current practices and existing challenges of risk analysis and management, e.g., its importance, identification of critical resources, involvement of different stakeholders, used methods, and follow-up analysis. To evaluate and compare the effectiveness of risk analysis methods we carried out a controlled experiment. In that study, we evaluated the effectiveness of risk analysis methods by counting the number of relevant and non-relevant risks identified by the experiment participants. The difficulty level of risk analysis methods and the experiment participants’ confidence about the identified risks were also investigated. Then, we carried out a case study to evaluate the effectiveness and efficiency of existing risk analysis methods, Failure Mode and Effect Analysis (FMEA) and System Theoretic Process Analysis (STPA). The case study investigates the effectiveness of the methods by performing a comparison of how a hazard analysis is conducted for the same system. It also evaluates the analysis process of risk analysis methods by using a set of qualitative criteria, derived from the Technology Acceptance Model (TAM). After this, another case study was carried out to evaluate and assess the resilience of critical IT systems and networks by applying a simulation method. A hybrid modeling approach was used which considers the technical network, represented using graph theory, as well as the repair system, represented by a queuing model. To improve the risk analysis process, this thesis also presents a new risk analysis method, Perspective Based Risk Analysis (PBRA), that uses different perspectives while analyzing IT systems. A perspective is a point of view or a specific role adopted by risk analyst while doing risk analysis, i.e., system engineer, system tester, or system user. Based on the findings, we conclude that the use of different perspectives improves effectiveness of risk analysis process. Then, to improve the risk analysis process we carried out a data mining study to save historical information about IT incidents to be used later for risk analysis. It could be an important aid in the process of building a database of occurred IT incidents that later can be used as an input to improve the risk analysis process. Finally, based on the findings of the studies included in this thesis a list of suggestions is presented to improve the risk analysis process. This list of potential suggestions was evaluated in a focus group meeting. The suggestions are for example, risk analysis awareness and education, defining clear roles and responsibilities, easy-to-use and adapt risk analysis methods, dealing with subjectivity, carry out risk analysis as early as possible and finally using historical risk data to improve the risk analysis process. Based on the findings it can be concluded that these suggestions are important and useful for risk practitioners to improve the risk analysis process.The presented research work in this thesis provides research about methods to improve the risk analysis and management practices. Moreover, the presented work in this thesis is based on solid empirical studies

Review and comparison of the modeling approaches and risk analysis methods for complex ship system.

Marine industry is leaning towards autonomous vessels with companies such as Rolls-Royce and Kongsberg leading the development. However, this rapid technological change invites greater risks and responsibilities for marine professionals. Ship systems are getting more complex with time as the interactions between components are increasing and software are getting embedded. As a result, the nature of risks in modern systems can be different than in the traditional systems, where the risks were mostly limited to human errors and component failures. However, for identifying risks in modern complex systems, it is first important to understand the structural composition of the system, and the component’s behavior, functions and interactions. Although, modern systems are quite different than traditional systems, traditional system-safety engineering techniques developed are still widely used. This thesis aims to review a modern modeling approach known as Systems Modeling Language (SysML) and a risk analysis method known as Systems-Theoretical Process Analysis (STPA); and compare them against widely used traditional methods known as the Tree structure method and Fault Tree Analysis (FTA). SysML, developed in 2006, is a graphical modeling language which presents structural composition, component functions, behavior, constraints and requirements of a system. SysML aims to support the analysis, specification, design, verification and validation of complex systems. STPA, developed in 2011, is a risk analysis method which aims to identify and mitigate risks in a complex system. Unlike traditional methods such as Fault Tree analysis (FTA), STPA focuses on risks due to the unsafe control actions and component interactions. Furthermore, STPA can be also used during the early phases of the system development process to generate safety constraints and requirements for a safer design of the system. This thesis also includes a workshop with Rolls-Royce where FTA, STPA, SysML and the Tree structure method were applied to a sample complex ship system. The results and feedback received from the workshop are presented and analyzed. The results suggest that the modern methods such as SysML and STPA are more suitable than traditional methods for modeling and identifying risks in a complex ship system if the results of the method’s implementation are considered. SysML presents several aspects of systems in a model which are missing in the Tree structure method, such as the requirements of a system, and behavior and interaction of components. Furthermore, it also provides a model that can be used as a tool for conducting an analysis of a system. Similarly, STPA succeeds on identifying higher number of risks related to component interactions and human errors in comparison to FTA, as STPA analyzes all possible control actions in a system, whereas FTA only analyzes the risks that are known to the analysts. However, some drawbacks of SysML and STPA have also been identified. Although the methods are suitable for complex ship systems, the methods have higher degree of complexity and require more time for an analysis in comparison to traditional methods. Furthermore, some solutions to improve the identified drawbacks of SysML and STPA are proposed in this thesis. Finally, some viable future research topics to improve the research results are presented

Review of the safety engineering techniques for a complex ship system

Marine industry is leaning towards the autonomous vessels; and advanced technologies are being developed for autonomous operations. However, this rapid technological change has increased the level of complexity in ship systems. As the interactions between components are increasing further and software are getting imbedded into components, the nature of risks in modern systems can be different than in the traditional systems; where the risks were mostly limited to human errors and component failures. However, for identifying risks in modern systems, it is first important to understand the system composition and the behavior of components. Since traditional system-safety engineering techniques, developed for the relatively simpler systems in past, are still dominant in marine industry. These techniques may not be able to cope with the risks due to increasing complexity.This paper reviews and identifies a suitable modelling approach and a risk analysis method for a complex ship system. A modern modeling approach known as Systems-Modeling Language (SysML) and a modern risk analysis method known as Systems-Theoretical Process Analysis (STPA) are reviewed and compared with widely used traditional methods known as the Tree structure method and Fault Tree Analysis. SysML is a graphical modeling language that presents structural composition, component functions, behavior, constraints and requirements of a complex system. STPA is a risk analysis method that aims to identify and mitigate risks in a complex system. The review and comparison results are presented in the paper.The results of this study suggest that the modern methods are more suitable than the traditional methods when the functionality of each method are considered. However, as the modern methods are more detailed, and are focused on the functionality, they are relatively complex and require more resources for the analysis in comparison to the traditional methods. Some viable solutions to improve the drawbacks of SysML and STPA, and possible future research topics are presented.Peer reviewe

From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML

Inappropriate design and deployment of machine learning (ML) systems leads to negative downstream social and ethical impact -- described here as social and ethical risks -- for users, society and the environment. Despite the growing need to regulate ML systems, current processes for assessing and mitigating risks are disjointed and inconsistent. We interviewed 30 industry practitioners on their current social and ethical risk management practices, and collected their first reactions on adapting safety engineering frameworks into their practice -- namely, System Theoretic Process Analysis (STPA) and Failure Mode and Effects Analysis (FMEA). Our findings suggest STPA/FMEA can provide appropriate structure toward social and ethical risk assessment and mitigation processes. However, we also find nontrivial challenges in integrating such frameworks in the fast-paced culture of the ML industry. We call on the ML research community to strengthen existing frameworks and assess their efficacy, ensuring that ML systems are safer for all people