A controlled experiment for the empirical evaluation of safety analysis techniques for safety-critical software

Context: Today's safety critical systems are increasingly reliant on software. Software becomes responsible for most of the critical functions of systems. Many different safety analysis techniques have been developed to identify hazards of systems. FTA and FMEA are most commonly used by safety analysts. Recently, STPA has been proposed with the goal to better cope with complex systems including software. Objective: This research aimed at comparing quantitatively these three safety analysis techniques with regard to their effectiveness, applicability, understandability, ease of use and efficiency in identifying software safety requirements at the system level. Method: We conducted a controlled experiment with 21 master and bachelor students applying these three techniques to three safety-critical systems: train door control, anti-lock braking and traffic collision and avoidance. Results: The results showed that there is no statistically significant difference between these techniques in terms of applicability, understandability and ease of use, but a significant difference in terms of effectiveness and efficiency is obtained. Conclusion: We conclude that STPA seems to be an effective method to identify software safety requirements at the system level. In particular, STPA addresses more different software safety requirements than the traditional techniques FTA and FMEA, but STPA needs more time to carry out by safety analysts with little or no prior experience.Comment: 10 pages, 1 figure in Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering (EASE '15). ACM, 201

Elicitation of structured engineering judgement to inform a focussed FMEA

The practical use of Failure Mode and Effects Analysis (FMEA) has been criticised because it is often implemented too late and in a manner that does not allow information to be fed-back to inform the product design. Lessons learnt from the use of elicitation methods to gather structured expert judgement about engineering concerns for a new product design has led to an enhancement of the approach for implementing design and process FMEA. We refer to this variant as a focussed FMEA since the goal is to enable relevant engineers to contribute to the analysis and to act upon the outcomes in such a way that all activities focus upon the design needs. The paper begins with a review of the proposed process to identify and quantify engineering concerns. The pros and cons of using elicitation methods, originally designed to support construction of a Bayesian prior, to inform a focussed FMEA are analysed and a comparison of the proposed process in relation to the existing standards is made. An industrial example is presented to illustrate customisation of the process and discuss the impact on the design process

Framework for continuous improvement of production processes

This research introduces a new approach of using Six Sigma DMAIC (Define, Measure, Analyse, Improve, Control) methodology. This approach integrates various tools and methods into a single framework, which consists of five steps. In the Define step, problems and main Key Performance Indicators (KPIs) are identified. In the Measure step, the modified Failure Classifier (FC), i.e. DOE-NE-STD-1004-92 is applied, which enables to specify the types of failures for each operation during the production process. Also, Failure Mode and Effect Analysis (FMEA) is used to measure the weight of failures by calculating the Risk Priority Number (RPN) value. In order to indicate the quality level of process/product the Process/Product Sigma Performance Level (PSPL) is calculated based on the FMEA results. Using the RPN values from FMEA the variability of process by failures, operations and work centres are observed. In addition, costs of the components are calculated, which enable to measure the impact of failures on the final product cost. A new method of analysis is introduced, in which various charts created in the Measure step are compared. Analysis step facilitates the subsequent Improve and Control steps, where appropriate changes in the manufacturing process are implemented and sustained. The objective of the new framework is to perform continuous improvement of production processes in the way that enables engineers to discover the critical problems that have financial impact on the final product. This framework provides new ways of monitoring and eliminating failures for production processes continuous improvement, by focusing on the KPIs important for business success. In this paper, the background and the key concepts of Six Sigma are described and the proposed Six Sigma DMAIC framework is explained. The implementation of this framework is verified by computational experiment followed by conclusion section

Integrating model checking with HiP-HOPS in model-based safety analysis

The ability to perform an effective and robust safety analysis on the design of modern safety–critical systems is crucial. Model-based safety analysis (MBSA) has been introduced in recent years to support the assessment of complex system design by focusing on the system model as the central artefact, and by automating the synthesis and analysis of failure-extended models. Model checking and failure logic synthesis and analysis (FLSA) are two prominent MBSA paradigms. Extensive research has placed emphasis on the development of these techniques, but discussion on their integration remains limited. In this paper, we propose a technique in which model checking and Hierarchically Performed Hazard Origin and Propagation Studies (HiP-HOPS) – an advanced FLSA technique – can be applied synergistically with benefit for the MBSA process. The application of the technique is illustrated through an example of a brake-by-wire system

Journal Staff

Riskanalysverktyget failure mode and effects analysis (FMEA) som analyserar kompo-nenter och signaler är beroende av att designen av fordonets elsystem finns tillgängligt och utförs därför sent i utvecklingsprocessen av elsystem. Detta medför att vissa fel inte analyseras i tid och kanske måste designas bort i efterhand vilket kan leda till ökad sy-stemkomplexitet samt längre och dyrare utvecklingsprocesser.Målet med examensarbetet är att genom en litteraturstudie ta reda på om det finns me-toder eller arbetssätt som gör att Scania tidigt i utvecklingsprocessen av elsystem kan genomföra funktionsanalyser i sitt riskanalysarbete med FMEA samt analysera dessa.Resultatet av detta examensarbete visar att det är möjligt att påbörja FMEA-arbetet tidigt i utvecklingsprocessen av elsystem om ingenjörerna utgår från ett funktionsperspektiv i riskanalysarbetet där de listar och rangordnar de funktioner som tillsammans realiserar en eller flera funktionaliteter samt deras felmoder, feleffekter, feldetektering, allvarlighet, sannolikhet och frekvens. Med hjälp av en FFMEA kan ingenjörerna tidigt i utveck-lingsprocessen av elsystem snabbt och effektivt hantera de identifierade säkerhetskritiska funktionerna.En befintlig funktionalitet på Scania har brutits ned i funktioner och en FFMEA har genomförts på dessa som en demonstration på hur en FFMEA kan genomföras och se ut.The risk analysis tool failure mode and effects analysis (FMEA) that analyzes the com-ponents and signals of a electrical system is design dependent and are therefore per-formed late in the development process of electrical systems. This could lead to that some errors are not analyzed in time and may need to be designed away which can lead to increased system complexity as well as longer and more expensive development proc-esses.The objective of this study is that through a literature review identify if there are any methods or approaches that enables Scania to implement a functional hazard analyzes early in the development process of electrical systems and to analyze these.The results of this thesis shows that it is possible to start the FMEA process early in the development process of the electrical system if the engineers have a functional perspec-tive in mind when performing the risk analysis where they list and rank the functions that is provided by the electrical system and their failure modes, failure effects, failure de-tection, severity, probability and occurrence.By using a function based FMEA, the engineer(s) can identify and promptly handle the safety critical functions early in the development process of a electrical system.A existing functionality at Scania has been broken down into functions and a functional hazard analysis has been performed on these as a demonstration of how a function based FMEA can be carried out and look like

Engineering failure analysis and design optimisation with HiP-HOPS

The scale and complexity of computer-based safety critical systems, like those used in the transport and manufacturing industries, pose significant challenges for failure analysis. Over the last decade, research has focused on automating this task. In one approach, predictive models of system failure are constructed from the topology of the system and local component failure models using a process of composition. An alternative approach employs model-checking of state automata to study the effects of failure and verify system safety properties. In this paper, we discuss these two approaches to failure analysis. We then focus on Hierarchically Performed Hazard Origin & Propagation Studies (HiP-HOPS) - one of the more advanced compositional approaches - and discuss its capabilities for automatic synthesis of fault trees, combinatorial Failure Modes and Effects Analyses, and reliability versus cost optimisation of systems via application of automatic model transformations. We summarise these contributions and demonstrate the application of HiP-HOPS on a simplified fuel oil system for a ship engine. In light of this example, we discuss strengths and limitations of the method in relation to other state-of-the-art techniques. In particular, because HiP-HOPS is deductive in nature, relating system failures back to their causes, it is less prone to combinatorial explosion and can more readily be iterated. For this reason, it enables exhaustive assessment of combinations of failures and design optimisation using computationally expensive meta-heuristics. (C) 2010 Elsevier Ltd. All rights reserved

Independent Orbiter Assessment (IOA): Assessment of the Electrical Power Distribution and Control/Electrical Power Generation (EPD and C/EPG) FMEA/CIL

The results of the Independent Orbiter Assessment (IOA) of the Failure Modes and Effects Analysis (FMEA) and Critical Items List (CIL) are presented. The IOA effort first completed an analysis of the Electrical Power Distribution and Control/Electrical Power Generation (EPD and C/EPG) hardware, generating draft failure modes and potential critical items. To preserve independence, this analysis was accomplished without reliance upon the results contained within the NASA FMEA/CIL documentation. The IOA results were then compared to the NASA FMEA/CIL baseline with proposed Post 51-L updates included. A resolution of each discrepancy from the comparison was provided through additional analysis as required. The results of that comparison is documented for the Orbiter EPD and C/EPG hardware. The IOA product for the EPD and C/EPG analysis consisted of 263 failure mode worksheets that resulted in 42 potential critical items being identified. Comparison was made to the NASA baseline which consisted of 211 FMEA and 47 CIL items