System importance measures: A new approach to resilient systems-of-systems

Resilience is the ability to withstand and recover rapidly from disruptions. While this attribute has been the focus of research in several fields, in the case of system-of-systems (SoSs), addressing resilience is particularly interesting and challenging. As infrastructure SoSs, such as power, transportation, and communication networks, grow in complexity and interconnectivity, measuring and improving the resilience of these SoSs is vital in terms of safety and providing uninterrupted services. ^ The characteristics of systems-of-systems make analysis and design of resilience challenging. However, these features also offer opportunities to make SoSs resilient using unconventional methods. In this research, we present a new approach to the process of resilience design. The core idea behind the proposed design process is a set of system importance measures (SIMs) that identify systems crucial to overall resilience. Using the results from the SIMs, we determine appropriate strategies from a list of design principles to improve SoS resilience. The main contribution of this research is the development of an aid to design that provides specific guidance on where and how resources need to be targeted. Based on the needs of an SoS, decision-makers can iterate through the design process to identify a set of practical and effective design improvements. ^ We use two case studies to demonstrate how the SIM-based design process can inform decision-making in the context of SoS resilience. The first case study focuses on a naval warfare SoS and describes how the resilience framework can leverage existing simulation models to support end-to-end design. We proceed through stages of the design approach using an agent-based model (ABM) that enables us to demonstrate how simulation tools and analytical models help determine the necessary inputs for the design process and, subsequently, inform decision-making regarding SoS resilience. ^ The second case study considers the urban transportation network in Boston. This case study focuses on interpreting the results of the resilience framework and on describing how they can be used to guide design choices in large infrastructure networks. We use different resilience maps to highlight the range of design-related information that can be obtained from the framework. ^ Specific advantages of the SIM-based resilience design include: (1) incorporates SoS- specific features within existing risk-based design processes - the SIMs determine the relative importance of different systems based on their impacts on SoS-level performance, and suggestions for resilience improvement draw from design options that leverage SoS- specific characteristics, such as the ability to adapt quickly (such as add new systems or re-task existing ones) and to provide partial recovery of performance in the aftermath of a disruption; (2) allows rapid understanding of different areas of concern within the SoS - the visual nature of the resilience map (a key outcome of the SIM analysis) provides a useful way to summarize the current resilience of the SoS as well as point to key systems of concern; and (3) provides a platform for multiple analysts and decision- makers to study, modify, discuss and documentoptions for SoS

Understanding, Assessing, and Mitigating Safety Risks in Artificial Intelligence Systems

Prepared for: Naval Air Warfare Development Center (NAVAIR)Traditional software safety techniques rely on validating software against a deductively defined specification of how the software should behave in particular situations. In the case of AI systems, specifications are often implicit or inductively defined. Data-driven methods are subject to sampling error since practical datasets cannot provide exhaustive coverage of all possible events in a real physical environment. Traditional software verification and validation approaches may not apply directly to these novel systems, complicating the operation of systems safety analysis (such as implemented in MIL-STD 882). However, AI offers advanced capabilities, and it is desirable to ensure the safety of systems that rely on these capabilities. When AI tech is deployed in a weapon system, robot, or planning system, unwanted events are possible. Several techniques can support the evaluation process for understanding the nature and likelihood of unwanted events in AI systems and making risk decisions on naval employment. This research considers the state of the art, evaluating which ones are most likely to be employable, usable, and correct. Techniques include software analysis, simulation environments, and mathematical determinations.Naval Air Warfare Development CenterNaval Postgraduate School, Naval Research Program (PE 0605853N/2098)Approved for public release. Distribution is unlimite

How to Think About Resilient Infrastructure Systems

abstract: Resilience is emerging as the preferred way to improve the protection of infrastructure systems beyond established risk management practices. Massive damages experienced during tragedies like Hurricane Katrina showed that risk analysis is incapable to prevent unforeseen infrastructure failures and shifted expert focus towards resilience to absorb and recover from adverse events. Recent, exponential growth in research is now producing consensus on how to think about infrastructure resilience centered on definitions and models from influential organizations like the US National Academy of Sciences. Despite widespread efforts, massive infrastructure failures in 2017 demonstrate that resilience is still not working, raising the question: Are the ways people think about resilience producing resilient infrastructure systems? This dissertation argues that established thinking harbors misconceptions about infrastructure systems that diminish attempts to improve their resilience. Widespread efforts based on the current canon focus on improving data analytics, establishing resilience goals, reducing failure probabilities, and measuring cascading losses. Unfortunately, none of these pursuits change the resilience of an infrastructure system, because none of them result in knowledge about how data is used, goals are set, or failures occur. Through the examination of each misconception, this dissertation results in practical, new approaches for infrastructure systems to respond to unforeseen failures via sensing, adapting, and anticipating processes. Specifically, infrastructure resilience is improved by sensing when data analytics include the modeler-in-the-loop, adapting to stress contexts by switching between multiple resilience strategies, and anticipating crisis coordination activities prior to experiencing a failure. Overall, results demonstrate that current resilience thinking needs to change because it does not differentiate resilience from risk. The majority of research thinks resilience is a property that a system has, like a noun, when resilience is really an action a system does, like a verb. Treating resilience as a noun only strengthens commitment to risk-based practices that do not protect infrastructure from unknown events. Instead, switching to thinking about resilience as a verb overcomes prevalent misconceptions about data, goals, systems, and failures, and may bring a necessary, radical change to the way infrastructure is protected in the future.Dissertation/ThesisDoctoral Dissertation Civil, Environmental and Sustainable Engineering 201

Review of emergent behaviours of systems comparable to infrastructure systemsand analysis approaches that could be applied to infrastructure systems

This paper makes contributions to the understanding of emergent failure in economic infrastructure by considering case studies and approaches from sectors comparable to infrastructure. The review starts by identifying existing ways of thinking about emergent failure and narrows down the scope to system-of-systems’ failures which are unexpected and arise when systems appear to be working normally. In order to target sectors similar to infrastructure, the characteristics of infrastructure sectors were characterised

Process Resilience Analysis Framework for Design and Operations

Process plants are complex socio-technical systems that degrade gradually and change with advancing technology. This research deals with exploring and answering questions related to the uncertainties involved in the process systems, and their complexity. It aims to systematically integrate resilience in process design and operations through three different phases of prediction, survival, and recovery using a novel framework called Process Resilience Analysis Framework (PRAF). The analysis relies on simulation, data-driven models and optimization approach employing the resilience metrics developed in this research. In particular, an integrated method incorporating aspects of process operations, equipment maintenance, and process safety is developed for the following three phases: •Prediction: to find the feasible operating region under changing conditions using Bayesian approach, global sensitivity analysis, and robust simulation methods, •Survival: to determine optimal operations and maintenance strategies using simulation, Bayesian regression analysis, and optimization, and •Recovery: to develop a strategy for emergency barriers in abnormal situations using dynamic simulation, Bayesian analysis, and optimization. Examples of a batch reactor, and cooling tower operations process unit are used to illustrate the application of PRAF. The results demonstrate that PRAF is successful in capturing the interactions between the process operability characteristics, maintenance, and safety policy. The prediction phase analysis leads to good dynamic response and stability of operations. The survival phase helps in the reduction of unplanned shutdown and downtime. The recovery phase results in in reduced severity of consequences, and response time and overall enhanced recovery. Overall, PRAF achieves flexibility, controllability and reliability of the system, supports more informed decision-making and profitable process systems

Advanced system engineering approaches to dynamic modelling of human factors and system safety in sociotechnical systems

Sociotechnical systems (STSs) indicate complex operational processes composed of interactive and dependent social elements, organizational and human activities. This research work seeks to fill some important knowledge gaps in system safety performance and human factors analysis using in STSs. First, an in-depth critical analysis is conducted to explore state-of-the-art findings, needs, gaps, key challenges, and research opportunities in human reliability and factors analysis (HR&FA). Accordingly, a risk model is developed to capture the dynamic nature of different systems failures and integrated them into system safety barriers under uncertainty as per Safety-I paradigm. This is followed by proposing a novel dynamic human-factor risk model tailored for assessing system safety in STSs based on Safety-II concepts. This work is extended to further explore system safety using Performance Shaping Factors (PSFs) by proposing a systematic approach to identify PSFs and quantify their importance level and influence on the performance of sociotechnical systems’ functions. Finally, a systematic review is conducted to provide a holistic profile of HR&FA in complex STSs with a deep focus on revealing the contribution of artificial intelligence and expert systems over HR&FA in complex systems. The findings reveal that proposed models can effectively address critical challenges associated with system safety and human factors quantification. It also trues about uncertainty characterization using the proposed models. Furthermore, the proposed advanced probabilistic model can better model evolving dependencies among system safety performance factors. It revealed the critical safety investment factors among different sociotechnical elements and contributing factors. This helps to effectively allocate safety countermeasures to improve resilience and system safety performance. This research work would help better understand, analyze, and improve the system safety and human factors performance in complex sociotechnical systems

Rapid Mission Assurance Assessment via Sociotechnical Modeling and Simulation

How do organizations rapidly assess command-level effects of cyber attacks? Leaders need a way of assuring themselves that their organization, people, and information technology can continue their missions in a contested cyber environment. To do this, leaders should: 1) require assessments be more than analogical, anecdotal or simplistic snapshots in time; 2) demand the ability to rapidly model their organizations; 3) identify their organization’s structural vulnerabilities; and 4) have the ability to forecast mission assurance scenarios. Using text mining to build agent based dynamic network models of information processing organizations, I examine impacts of contested cyber environments on three common focus areas of information assurance—confidentiality, integrity, and availability. I find that assessing impacts of cyber attacks is a nuanced affair dependent on the nature of the attack, the nature of the organization and its missions, and the nature of the measurements. For well-manned information processing organizations, many attacks are in the nuisance range and that only multipronged or severe attacks cause meaningful failure. I also find that such organizations can design for resiliency and provide guidelines in how to do so