Transient Behaviour in Highly Dependable Markovian Systems: New Regimes, Multiple Paths

In recent years, probabilistic analysis of highly dependable Markovian systems has received considerable attention. Such systems typically consist of several component types, subject to failures, with spare components for replacement while repair is taking place. System failure occurs when all (spare) components of one or several types have failed. In this work we try to estimate the probability of system failure before some fixed time bound $\tau$ via stochastic simulation. Obviously, in a highly dependable system, system failure is a rare event, so we apply importance sampling (IS) techniques, based on knowledge of the behaviour of the system and the way the rare event occurs. In our talk we discern several interesting ways in which the rare event can occur, each of which has its own way of affecting the efficiency of an importance sampling technique

Spacecraft dynamics characterization and control system failure detection

Two important aspects of the control of large space structures are studied: the modeling of deployed or erected structures including nonlinear joint characteristics; and the detection and isolation of failures of the components of control systems for large space structures. The emphasis in the first task is on efficient representation of the dynamics of large and complex structures having a great many joints. The initial emphasis in the second task is on experimental evaluation of FDI methodologies using ground-based facilities in place at NASA Langley Research Center and Marshall Space Flight Center. The progress to date on both research tasks is summarized

Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure

As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggregate views of system performance that hide important shortcomings. Understanding details about failures is important for identifying pathways for refinement, communicating the reliability of systems in different settings, and for specifying appropriate human oversight and engagement. Characterization of failures and shortcomings is particularly complex for systems composed of multiple machine learned components. For such systems, existing evaluation methods have limited expressiveness in describing and explaining the relationship among input content, the internal states of system components, and final output quality. We present Pandora, a set of hybrid human-machine methods and tools for describing and explaining system failures. Pandora leverages both human and system-generated observations to summarize conditions of system malfunction with respect to the input content and system architecture. We share results of a case study with a machine learning pipeline for image captioning that show how detailed performance views can be beneficial for analysis and debugging

Pilot dynamic response to sudden flight control system failures and implications for design

Pilot dynamic response to sudden flight control system failure

Reliability growth during a development testing program

Binomial and trinomial mathematical models for reliability growth studies - statistical analysis of system failure

Space tug propulsion system failure mode, effects and criticality analysis

For purposes of the study, the propulsion system was considered as consisting of the following: (1) main engine system, (2) auxiliary propulsion system, (3) pneumatic system, (4) hydrogen feed, fill, drain and vent system, (5) oxygen feed, fill, drain and vent system, and (6) helium reentry purge system. Each component was critically examined to identify possible failure modes and the subsequent effect on mission success. Each space tug mission consists of three phases: launch to separation from shuttle, separation to redocking, and redocking to landing. The analysis considered the results of failure of a component during each phase of the mission. After the failure modes of each component were tabulated, those components whose failure would result in possible or certain loss of mission or inability to return the Tug to ground were identified as critical components and a criticality number determined for each. The criticality number of a component denotes the number of mission failures in one million missions due to the loss of that component. A total of 68 components were identified as critical with criticality numbers ranging from 1 to 2990