36,420 research outputs found
On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection
Humans are the final decision makers in critical tasks that involve ethical
and legal concerns, ranging from recidivism prediction, to medical diagnosis,
to fighting against fake news. Although machine learning models can sometimes
achieve impressive performance in these tasks, these tasks are not amenable to
full automation. To realize the potential of machine learning for improving
human decisions, it is important to understand how assistance from machine
learning models affects human performance and human agency.
In this paper, we use deception detection as a testbed and investigate how we
can harness explanations and predictions of machine learning models to improve
human performance while retaining human agency. We propose a spectrum between
full human agency and full automation, and develop varying levels of machine
assistance along the spectrum that gradually increase the influence of machine
predictions. We find that without showing predicted labels, explanations alone
slightly improve human performance in the end task. In comparison, human
performance is greatly improved by showing predicted labels (>20% relative
improvement) and can be further improved by explicitly suggesting strong
machine performance. Interestingly, when predicted labels are shown,
explanations of machine predictions induce a similar level of accuracy as an
explicit statement of strong machine performance. Our results demonstrate a
tradeoff between human performance and human agency and show that explanations
of machine predictions can moderate this tradeoff.Comment: 17 pages, 19 figures, in Proceedings of ACM FAT* 2019, dataset & demo
available at https://deception.machineintheloop.co
Explanation in Human-AI Systems: A Literature Meta-Review, Synopsis of Key Ideas and Publications, and Bibliography for Explainable AI
This is an integrative review that address the question, "What makes for a
good explanation?" with reference to AI systems. Pertinent literatures are
vast. Thus, this review is necessarily selective. That said, most of the key
concepts and issues are expressed in this Report. The Report encapsulates the
history of computer science efforts to create systems that explain and instruct
(intelligent tutoring systems and expert systems). The Report expresses the
explainability issues and challenges in modern AI, and presents capsule views
of the leading psychological theories of explanation. Certain articles stand
out by virtue of their particular relevance to XAI, and their methods, results,
and key points are highlighted. It is recommended that AI/XAI researchers be
encouraged to include in their research reports fuller details on their
empirical or experimental methods, in the fashion of experimental psychology
research reports: details on Participants, Instructions, Procedures, Tasks,
Dependent Variables (operational definitions of the measures and metrics),
Independent Variables (conditions), and Control Conditions
How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation
Recent years have seen a boom in interest in machine learning systems that
can provide a human-understandable rationale for their predictions or
decisions. However, exactly what kinds of explanation are truly
human-interpretable remains poorly understood. This work advances our
understanding of what makes explanations interpretable in the specific context
of verification. Suppose we have a machine learning system that predicts X, and
we provide rationale for this prediction X. Given an input, an explanation, and
an output, is the output consistent with the input and the supposed rationale?
Via a series of user-studies, we identify what kinds of increases in complexity
have the greatest effect on the time it takes for humans to verify the
rationale, and which seem relatively insensitive
LIMEtree: Interactively Customisable Explanations Based on Local Surrogate Multi-output Regression Trees
Systems based on artificial intelligence and machine learning models should
be transparent, in the sense of being capable of explaining their decisions to
gain humans' approval and trust. While there are a number of explainability
techniques that can be used to this end, many of them are only capable of
outputting a single one-size-fits-all explanation that simply cannot address
all of the explainees' diverse needs. In this work we introduce a
model-agnostic and post-hoc local explainability technique for black-box
predictions called LIMEtree, which employs surrogate multi-output regression
trees. We validate our algorithm on a deep neural network trained for object
detection in images and compare it against Local Interpretable Model-agnostic
Explanations (LIME). Our method comes with local fidelity guarantees and can
produce a range of diverse explanation types, including contrastive and
counterfactual explanations praised in the literature. Some of these
explanations can be interactively personalised to create bespoke, meaningful
and actionable insights into the model's behaviour. While other methods may
give an illusion of customisability by wrapping, otherwise static, explanations
in an interactive interface, our explanations are truly interactive, in the
sense of allowing the user to "interrogate" a black-box model. LIMEtree can
therefore produce consistent explanations on which an interactive exploratory
process can be built
The Grammar of Interactive Explanatory Model Analysis
The growing need for in-depth analysis of predictive models leads to a series
of new methods for explaining their local and global properties. Which of these
methods is the best? It turns out that this is an ill-posed question. One
cannot sufficiently explain a black-box machine learning model using a single
method that gives only one perspective. Isolated explanations are prone to
misunderstanding, which inevitably leads to wrong or simplistic reasoning. This
problem is known as the Rashomon effect and refers to diverse, even
contradictory interpretations of the same phenomenon. Surprisingly, the
majority of methods developed for explainable machine learning focus on a
single aspect of the model behavior. In contrast, we showcase the problem of
explainability as an interactive and sequential analysis of a model. This paper
presents how different Explanatory Model Analysis (EMA) methods complement each
other and why it is essential to juxtapose them together. The introduced
process of Interactive EMA (IEMA) derives from the algorithmic side of
explainable machine learning and aims to embrace ideas developed in cognitive
sciences. We formalize the grammar of IEMA to describe potential human-model
dialogues. IEMA is implemented in the human-centered framework that adopts
interactivity, customizability and automation as its main traits. Combined,
these methods enhance the responsible approach to predictive modeling.Comment: 17 pages, 10 figures, 3 table
An Interaction Framework for Studying Co-Creative AI
Machine learning has been applied to a number of creative, design-oriented
tasks. However, it remains unclear how to best empower human users with these
machine learning approaches, particularly those users without technical
expertise. In this paper we propose a general framework for turn-based
interaction between human users and AI agents designed to support human
creativity, called {co-creative systems}. The framework can be used to better
understand the space of possible designs of co-creative systems and reveal
future research directions. We demonstrate how to apply this framework in
conjunction with a pair of recent human subject studies, comparing between the
four human-AI systems employed in these studies and generating hypotheses
towards future studies.Comment: 6 pages, 2 figures, Human-Centered Machine Learning Perspectives
Worksho
A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations
Human-in-the-loop data analysis applications necessitate greater transparency
in machine learning models for experts to understand and trust their decisions.
To this end, we propose a visual analytics workflow to help data scientists and
domain experts explore, diagnose, and understand the decisions made by a binary
classifier. The approach leverages "instance-level explanations", measures of
local feature relevance that explain single instances, and uses them to build a
set of visual representations that guide the users in their investigation. The
workflow is based on three main visual representations and steps: one based on
aggregate statistics to see how data distributes across correct / incorrect
decisions; one based on explanations to understand which features are used to
make these decisions; and one based on raw data, to derive insights on
potential root causes for the observed patterns. The workflow is derived from a
long-term collaboration with a group of machine learning and healthcare
professionals who used our method to make sense of machine learning models they
developed. The case study from this collaboration demonstrates that the
proposed workflow helps experts derive useful knowledge about the model and the
phenomena it describes, thus experts can generate useful hypotheses on how a
model can be improved.Comment: Published at IEEE Conference on Visual Analytics Science and
Technology (IEEE VAST 2017
"I had a solid theory before but it's falling apart": Polarizing Effects of Algorithmic Transparency
The rise of machine learning has brought closer scrutiny to intelligent
systems, leading to calls for greater transparency and explainable algorithms.
We explore the effects of transparency on user perceptions of a working
intelligent system for emotion detection. In exploratory Study 1, we observed
paradoxical effects of transparency which improves perceptions of system
accuracy for some participants while reducing accuracy perceptions for others.
In Study 2, we test this observation using mixed methods, showing that the
apparent transparency paradox can be explained by a mismatch between
participant expectations and system predictions. We qualitatively examine this
process, indicating that transparency can undermine user confidence by causing
users to fixate on flaws when they already have a model of system operation. In
contrast transparency helps if users lack such a model. Finally, we revisit the
notion of transparency and suggest design considerations for building safe and
successful machine learning systems based on our insights
Explainability in Human-Agent Systems
This paper presents a taxonomy of explainability in Human-Agent Systems. We
consider fundamental questions about the Why, Who, What, When and How of
explainability. First, we define explainability, and its relationship to the
related terms of interpretability, transparency, explicitness, and
faithfulness. These definitions allow us to answer why explainability is needed
in the system, whom it is geared to and what explanations can be generated to
meet this need. We then consider when the user should be presented with this
information. Last, we consider how objective and subjective measures can be
used to evaluate the entire system. This last question is the most encompassing
as it will need to evaluate all other issues regarding explainability
Interactive Data Integration through Smart Copy & Paste
In many scenarios, such as emergency response or ad hoc collaboration, it is
critical to reduce the overhead in integrating data. Ideally, one could perform
the entire process interactively under one unified interface: defining
extractors and wrappers for sources, creating a mediated schema, and adding
schema mappings ? while seeing how these impact the integrated view of the
data, and refining the design accordingly.
We propose a novel smart copy and paste (SCP) model and architecture for
seamlessly combining the design-time and run-time aspects of data integration,
and we describe an initial prototype, the CopyCat system. In CopyCat, the user
does not need special tools for the different stages of integration: instead,
the system watches as the user copies data from applications (including the Web
browser) and pastes them into CopyCat?s spreadsheet-like workspace. CopyCat
generalizes these actions and presents proposed auto-completions, each with an
explanation in the form of provenance. The user provides feedback on these
suggestions ? through either direct interactions or further copy-and-paste
operations ? and the system learns from this feedback. This paper provides an
overview of our prototype system, and identifies key research challenges in
achieving SCP in its full generality.Comment: CIDR 200
- …