On Evidence-based Risk Management in Requirements Engineering

Background: The sensitivity of Requirements Engineering (RE) to the context makes it difficult to efficiently control problems therein, thus, hampering an effective risk management devoted to allow for early corrective or even preventive measures. Problem: There is still little empirical knowledge about context-specific RE phenomena which would be necessary for an effective context- sensitive risk management in RE. Goal: We propose and validate an evidence-based approach to assess risks in RE using cross-company data about problems, causes and effects. Research Method: We use survey data from 228 companies and build a probabilistic network that supports the forecast of context-specific RE phenomena. We implement this approach using spreadsheets to support a light-weight risk assessment. Results: Our results from an initial validation in 6 companies strengthen our confidence that the approach increases the awareness for individual risk factors in RE, and the feedback further allows for disseminating our approach into practice.Comment: 20 pages, submitted to 10th Software Quality Days conference, 201

NASA Human System Risk Assessment Process

NASA utilizes an evidence based system to perform risk assessments for the human system for spaceflight missions. The center of this process is the multi-disciplinary Human System Risk Board (HSRB). The HSRB is chartered from the Chief Health and Medical Officer (OCHMO) at NASA Headquarters. The HSRB reviews all human system risks via an established comprehensive risk and configuration management plan based on a project management approach. The HSRB facilitates the integration of human research (terrestrial and spaceflight), medical operations, occupational surveillance, systems engineering and many other disciplines in a comprehensive review of human system risks. The HSRB considers all factors that influence human risk. These factors include pre-mission considerations such as screening criteria, training, age, sex, and physiological condition. In mission factors such as available countermeasures, mission duration and location and post mission factors such as time to return to baseline (reconditioning), post mission health screening, and available treatments. All of the factors influence the total risk assessment for each human risk. The HSRB performed a comprehensive review of all potential inflight medical conditions and events and over the course of several reviews consolidated the number of human system risks to 30, where the greatest emphasis is placed for investing program dollars for risk mitigation. The HSRB considers all available evidence from human research and, medical operations and occupational surveillance in assessing the risks for appropriate mitigation and future work. All applicable DRMs (low earth orbit for 6 and 12 months, deep space for 30 days and 1 year, a lunar mission for 1 year, and a planetary mission for 3 years) are considered as human system risks are modified by the hazards associated with space flight such as microgravity, exposure to radiation, distance from the earth, isolation and a closed environment. Each risk has a summary two-page assessment representing the state of knowledge/evidence of that risk, available risk mitigations, traceability to the Space Flight Human System Standards (SFHSS) and program requirements, and future work required. These data then can drive coordinated budgets across the Human Research Program, the International Space Station, Crew Health and Safety and Advanced Exploration System budgets to provide the most economical and timely mitigations. The risk assessments were completed for the 6 DRMs and serve as the baseline for which subsequent research and technology development and crew health care portfolios can be assessed. The HSRB reviews each risk at least annually or when new evidence/information is available that adds to the body of evidence. The current status of each risk can be reported to program management for operations, budget reviews and general oversight of the human system risk management program

Quality assurance and risk management: Perspectives on Human Factors Certification of Advanced Aviation Systems

This paper is based on the experience of engineering psychologists advising the U.K. Ministry of Defense (MoD) on the procurement of advanced aviation systems that conform to good human engineering (HE) practice. Traditional approaches to HE in systems procurement focus on the physical nature of the human-machine interface. Advanced aviation systems present increasingly complex design requirements for human functional integration, information processing, and cognitive task performance effectiveness. These developing requirements present new challenges for HE quality assurance (QA) and risk management, requiring focus on design processes as well as on design content or product. A new approach to the application of HE, recently adopted by NATO, provides more systematic ordering and control of HE processes and activities to meet the challenges of advanced aircrew systems design. This systematic approach to HE has been applied by MoD to the procurement of mission systems for the Royal Navy Merlin helicopter. In MoD procurement, certification is a judicial function, essentially independent of the service customer and industry contractor. Certification decisions are based on advice from MoD's appointed Acceptance Agency. Test and evaluation (T&E) conducted by the contractor and by the Acceptance Agency provide evidence for certification. Certification identifies limitations of systems upon release to the service. Evidence of compliance with HE standards traditionally forms the main basis of HE certification and significant non-compliance could restrict release. The systems HE approach shows concern for the quality of processes as well as for the content of the product. Human factors certification should be concerned with the quality of HE processes as well as products. Certification should require proof of process as well as proof of content and performance. QA criteria such as completeness, consistency, timeliness, and compatibility provide generic guidelines for progressive acceptance and certification of HE processes. Threats to the validity of certification arise from problems and assumptions in T&E methods. T&E should seek to reduce the risk of specification non-compliance and certification failure

Why We Cannot (Yet) Ensure the Cybersecurity of Safety-Critical Systems

There is a growing threat to the cyber-security of safety-critical systems. The introduction of Commercial Off The Shelf (COTS) software, including Linux, specialist VOIP applications and Satellite Based Augmentation Systems across the aviation, maritime, rail and power-generation infrastructures has created common, vulnerabilities. In consequence, more people now possess the technical skills required to identify and exploit vulnerabilities in safety-critical systems. Arguably for the first time there is the potential for cross-modal attacks leading to future ‘cyber storms’. This situation is compounded by the failure of public-private partnerships to establish the cyber-security of safety critical applications. The fiscal crisis has prevented governments from attracting and retaining competent regulators at the intersection of safety and cyber-security. In particular, we argue that superficial similarities between safety and security have led to security policies that cannot be implemented in safety-critical systems. Existing office-based security standards, such as the ISO27k series, cannot easily be integrated with standards such as IEC61508 or ISO26262. Hybrid standards such as IEC 62443 lack credible validation. There is an urgent need to move beyond high-level policies and address the more detailed engineering challenges that threaten the cyber-security of safety-critical systems. In particular, we consider the ways in which cyber-security concerns undermine traditional forms of safety engineering, for example by invalidating conventional forms of risk assessment. We also summarise the ways in which safety concerns frustrate the deployment of conventional mechanisms for cyber-security, including intrusion detection systems

Using software development progress data to understand threats to project outcomes

Peer reviewe

Safety Engineering with COTS components

Safety-critical systems are becoming more widespread, complex and reliant on software. Increasingly they are engineered through Commercial Off The Shelf (COTS) (Commercial Off The Shelf) components to alleviate the spiralling costs and development time, often in the context of complex supply chains. A parallel increased concern for safety has resulted in a variety of safety standards, with a growing consensus that a safety life cycle is needed which is fully integrated with the design and development life cycle, to ensure that safety has appropriate influence on the design decisions as system development progresses. In this article we explore the application of an integrated approach to safety engineering in which assurance drives the engineering process. The paper re- ports on the outcome of a case study on a live industrial project with a view to evaluate: its suitability for application in a real-world safety engineering setting; its benefits and limitations in counteracting some of the difficulties of safety en- gineering with COTS components across supply chains; and, its effectiveness in generating evidence which can contribute directly to the construction of safety cases

Environmental (waste) compliance control systems for UK SMEs

While the ‘environment’ is often perceived as a heavily regulated area of business, in reality, directly-regulated businesses represent a small proportion of the business community. This study aimed to evaluate and outline potential improvements to compliance controls for small and medium-sized enterprises (SMEs), particularly those involved in the waste sector. Forty-four SMEs from England were interviewed/audited between April-September 2008. Using a UK-based system as a case-in-point, the Environment Agency’s (EA) Operational Risk Appraisal (‘Opra’)/Compliance Assessment Report (CAR) system was analysed. Environmental compliance performance indicators and an initial assessment methodology for SMEs were developed. The study showed:• Compliance with permitting legislation was poor in many areas.• Regulatory authorities are either unable/failing to implement their enforcement policies or unable/failing to identify non-compliances due to the infrequency or limited nature of their inspections.• Improvements are needed to the EA Opra/CAR system – control measures are not fully taken into account when calculating risk.Recommendations to improve SME compliance controls include using internationally applicable general and specific compliance and non-compliance performance indicators, re-designing the Opra system and using an initial assessment methodology based on understanding the hazardousness of SME categories, compliance levels and operator competency.<br/

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

Intellectual Property Topics in Open University Distance-Taught Courses

Patents lie at the heart of engineering as a permanent and ongoing record of invention. We have taught the subject for about 5 years in both UG and PG courses, written from scratch owing to the absence of textbooks aimed specifically at engineers. Most practising engineers develop patent skills on the job rather than through conventional courses. But there is a need to present such courses as early as possible in the engineering curriculum, so that graduates have a flying start in their first employment