Towards Accountable AI: Hybrid Human-Machine Analyses for Characterizing System Failure

As machine learning systems move from computer-science laboratories into the open world, their accountability becomes a high priority problem. Accountability requires deep understanding of system behavior and its failures. Current evaluation methods such as single-score error metrics and confusion matrices provide aggregate views of system performance that hide important shortcomings. Understanding details about failures is important for identifying pathways for refinement, communicating the reliability of systems in different settings, and for specifying appropriate human oversight and engagement. Characterization of failures and shortcomings is particularly complex for systems composed of multiple machine learned components. For such systems, existing evaluation methods have limited expressiveness in describing and explaining the relationship among input content, the internal states of system components, and final output quality. We present Pandora, a set of hybrid human-machine methods and tools for describing and explaining system failures. Pandora leverages both human and system-generated observations to summarize conditions of system malfunction with respect to the input content and system architecture. We share results of a case study with a machine learning pipeline for image captioning that show how detailed performance views can be beneficial for analysis and debugging

Trusting outgroup, but not ingroup members, requires control: neural and behavioral evidence.

Trust and cooperation often break down across group boundaries, contributing to pernicious consequences, from polarized political structures to intractable conflict. As such, addressing such conflicts require first understanding why trust is reduced in intergroup settings. Here, we clarify the structure of intergroup trust using neuroscientific and behavioral methods. We found that trusting ingroup members produced activity in brain areas associated with reward, whereas trusting outgroup members produced activity in areas associated with top-down control. Behaviorally, time pressure-which reduces people's ability to exert control-reduced individuals' trust in outgroup, but not ingroup members. These data suggest that the exertion of control can help recover trust in intergroup settings, offering potential avenues for reducing intergroup failures in trust and the consequences of these failures

Failure of non-vacuum steam sterilization processes for dental handpieces

Background: Dental handpieces are used in critical and semi-critical operative interventions. Although a number of dental professional bodies recommend that dental handpieces are sterilized between patient use there is a lack of clarity and understanding of the effectiveness of different steam sterilization processes. The internal mechanisms of dental handpieces contain narrow lumens (0·8-2·3mm) which can impede the removal of air and ingress of saturated steam required to achieve sterilization conditions. Aim: To identify the extent of sterilization failure in dental handpieces using a non-vacuum process. Methods: In-vitro and in-vivo investigations were conducted on commonly used UK benchtop steam sterilizers and three different types of dental handpieces. The sterilization process was monitored inside the lumens of dental handpieces using thermometric (TM) methods (dataloggers), chemical indicators (CI) and biological indicators (BI). Findings: All three methods of assessing achievement of sterility within dental handpieces that had been exposed to non-vacuum sterilization conditions demonstrated a significant number of failures (CI=8/3,024(fails/n tests); BI=15/3,024; TM=56/56) compared to vacuum sterilization conditions (CI=2/1,944; BI=0/1,944; TM=0/36). The dental handpiece most likely to fail sterilization in the non-vacuum process was the surgical handpiece. Non-vacuum sterilizers located in general dental practice had a higher rate of sterilization failure (CI=25/1,620; BI=32/1,620; TM=56/56) with no failures in vacuum process. Conclusion: Non-vacuum downward/gravity displacement, type-N steam sterilizers are an unreliable method for sterilization of dental handpieces in general dental practice. The handpiece most likely to fail sterilization is the type most frequently used for surgical interventions

Identification of the human factors contributing to maintenance failures in a petroleum operation

Objective: This research aimed to identify the most frequently occurring human factors contributing to maintenance-related failures within a petroleum industry organization. Commonality between failures will assist in understanding reliability in maintenance processes, thereby preventing accidents in high-hazard domains. Background: Methods exist for understanding the human factors contributing to accidents. Their application in a maintenance context mainly has been advanced in aviation and nuclear power. Maintenance in the petroleum industry provides a different context for investigating the role that human factors play in influencing outcomes. It is therefore worth investigating the contributing human factors to improve our understanding of both human factors in reliability and the factors specific to this domain. Method: Detailed analyses were conducted of maintenance- related failures (N = 38) in a petroleum company using structured interviews with maintenance technicians. The interview structure was based on the Human Factor Investigation Tool (HFIT), which in turn was based on Rasmussen’s model of human malfunction .Results: A mean of 9.5 factors per incident was identified across the cases investigated. The three most frequent human factors contributing to the maintenance failures were found to be assumption (79% of cases), design and maintenance (71%), and communication (66%).Conclusion: HFIT proved to be a useful instrument for identifying the pattern of human factors that recurred most frequently in maintenance-related failures. Application: The high frequency of failures attributed to assumptions and communication demonstrated the importance of problem-solving abilities and organizational communication in a domain where maintenance personnel have a high degree of autonomy and a wide geographical distribution

Explainable and Interpretable Decision-Making for Robotic Tasks

Future generations of robots, such as service robots that support humans with household tasks, will be a pervasive part of our daily lives. The human\u27s ability to understand the decision-making process of robots is thereby considered to be crucial for establishing trust-based and efficient interactions between humans and robots. In this thesis, we present several interpretable and explainable decision-making methods that aim to improve the human\u27s understanding of a robot\u27s actions, with a particular focus on the explanation of why robot failures were committed.In this thesis, we consider different types of failures, such as task recognition errors and task execution failures. Our first goal is an interpretable approach to learning from human demonstrations (LfD), which is essential for robots to learn new tasks without the time-consuming trial-and-error learning process. Our proposed method deals with the challenge of transferring human demonstrations to robots by an automated generation of symbolic planning operators based on interpretable decision trees. Our second goal is the prediction, explanation, and prevention of robot task execution failures based on causal models of the environment. Our contribution towards the second goal is a causal-based method that finds contrastive explanations for robot execution failures, which enables robots to predict, explain and prevent even timely shifted action failures (e.g., the current action was successful but will negatively affect the success of future actions). Since learning causal models is data-intensive, our final goal is to improve the data efficiency by utilizing prior experience. This investigation aims to help robots learn causal models faster, enabling them to provide failure explanations at the cost of fewer action execution experiments.In the future, we will work on scaling up the presented methods to generalize to more complex, human-centered applications

Evolution of Threats in the Global Risk Network

With a steadily growing population and rapid advancements in technology, the global economy is increasing in size and complexity. This growth exacerbates global vulnerabilities and may lead to unforeseen consequences such as global pandemics fueled by air travel, cyberspace attacks, and cascading failures caused by the weakest link in a supply chain. Hence, a quantitative understanding of the mechanisms driving global network vulnerabilities is urgently needed. Developing methods for efficiently monitoring evolution of the global economy is essential to such understanding. Each year the World Economic Forum publishes an authoritative report on the state of the global economy and identifies risks that are likely to be active, impactful or contagious. Using a Cascading Alternating Renewal Process approach to model the dynamics of the global risk network, we are able to answer critical questions regarding the evolution of this network. To fully trace the evolution of the network we analyze the asymptotic state of risks (risk levels which would be reached in the long term if the risks were left unabated) given a snapshot in time, this elucidates the various challenges faced by the world community at each point in time. We also investigate the influence exerted by each risk on others. Results presented here are obtained through either quantitative analysis or computational simulations.Comment: 27 pages, 15 figure

The dimensions of therapists\u27 thoughts in response to therapy failures.

This research study has explored the kinds of thoughts that therapists report having had in response to their experiences with therapy failures. The central goal was to develop a model for organizing therapists\u27 thoughts to form a basis for further investigations into therapists\u27 conceptual processes for coping with and learning from therapy failures. The methodological approaches used in this study were designed to conform to a set of hermeneutic and social constructionist assumptions about the development and function of meaning making, as it applies to both psychological research and the therapeutic relationship. Thus, the research methods replicated a social construction process, using a community of participants for all stages of data gathering and analyses. The application of Thought Listing and Multiple Sorting Procedures in combination with Cluster and Multidimensional Scaling Analyses yielded a three dimensional solution with which to organize these therapists\u27 thoughts. Additional findings suggest that the ways in which therapists examine therapy failures is socially constructed and may function to preserve therapists\u27 core beliefs. The three dimensional solution challenges the usefulness of an exclusively causal model for understanding therapists\u27 reflections on failures