4 research outputs found
Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience
The wide adoption of Machine Learning technologies has created a rapidly
growing demand for people who can train ML models. Some advocated the term
"machine teacher" to refer to the role of people who inject domain knowledge
into ML models. One promising learning paradigm is Active Learning (AL), by
which the model intelligently selects instances to query the machine teacher
for labels. However, in current AL settings, the human-AI interface remains
minimal and opaque. We begin considering AI explanations as a core element of
the human-AI interface for teaching machines. When a human student learns, it
is a common pattern to present one's own reasoning and solicit feedback from
the teacher. When a ML model learns and still makes mistakes, the human teacher
should be able to understand the reasoning underlying the mistakes. When the
model matures, the machine teacher should be able to recognize its progress in
order to trust and feel confident about their teaching outcome. Toward this
vision, we propose a novel paradigm of explainable active learning (XAL), by
introducing techniques from the recently surging field of explainable AI (XAI)
into an AL setting. We conducted an empirical study comparing the model
learning outcomes, feedback content and experience with XAL, to that of
traditional AL and coactive learning (providing the model's prediction without
the explanation). Our study shows benefits of AI explanation as interfaces for
machine teaching--supporting trust calibration and enabling rich forms of
teaching feedback, and potential drawbacks--anchoring effect with the model
judgment and cognitive workload. Our study also reveals important individual
factors that mediate a machine teacher's reception to AI explanations,
including task knowledge, AI experience and need for cognition. By reflecting
on the results, we suggest future directions and design implications for XAL.Comment: replacing with a draft accepted to CSCW202
Active Learning++: Incorporating Annotator's Rationale using Local Model Explanation
We propose a new active learning (AL) framework, Active Learning++, which can
utilize an annotator's labels as well as its rationale. Annotators can provide
their rationale for choosing a label by ranking input features based on their
importance for a given query. To incorporate this additional input, we modified
the disagreement measure for a bagging-based Query by Committee (QBC) sampling
strategy. Instead of weighing all committee models equally to select the next
instance, we assign higher weight to the committee model with higher agreement
with the annotator's ranking. Specifically, we generated a feature
importance-based local explanation for each committee model. The similarity
score between feature rankings provided by the annotator and the local model
explanation is used to assign a weight to each corresponding committee model.
This approach is applicable to any kind of ML model using model-agnostic
techniques to generate local explanation such as LIME. With a simulation study,
we show that our framework significantly outperforms a QBC based vanilla AL
framework.Comment: Accepted at Workshop on Data Science with Human in the Loop (DaSH) @
ACM SIGKDD 202
ALEX: Active Learning based Enhancement of a Model's Explainability
An active learning (AL) algorithm seeks to construct an effective classifier
with a minimal number of labeled examples in a bootstrapping manner. While
standard AL heuristics, such as selecting those points for annotation for which
a classification model yields least confident predictions, there has been no
empirical investigation to see if these heuristics lead to models that are more
interpretable to humans. In the era of data-driven learning, this is an
important research direction to pursue. This paper describes our
work-in-progress towards developing an AL selection function that in addition
to model effectiveness also seeks to improve on the interpretability of a model
during the bootstrapping steps. Concretely speaking, our proposed selection
function trains an `explainer' model in addition to the classifier model, and
favours those instances where a different part of the data is used, on an
average, to explain the predicted class. Initial experiments exhibited
encouraging trends in showing that such a heuristic can lead to developing more
effective and more explainable end-to-end data-driven classifiers.Comment: CIKM 202
Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Making
Although AI holds promise for improving human decision making in societally
critical domains, it remains an open question how human-AI teams can reliably
outperform AI alone and human alone in challenging prediction tasks (also known
as complementary performance). We explore two directions to understand the gaps
in achieving complementary performance. First, we argue that the typical
experimental setup limits the potential of human-AI teams. To account for lower
AI performance out-of-distribution than in-distribution because of distribution
shift, we design experiments with different distribution types and investigate
human performance for both in-distribution and out-of-distribution examples.
Second, we develop novel interfaces to support interactive explanations so that
humans can actively engage with AI assistance. Using virtual pilot studies and
large-scale randomized experiments across three tasks, we demonstrate a clear
difference between in-distribution and out-of-distribution, and observe mixed
results for interactive explanations: while interactive explanations improve
human perception of AI assistance's usefulness, they may reinforce human biases
and lead to limited performance improvement. Overall, our work points out
critical challenges and future directions towards enhancing human performance
with AI assistance.Comment: 43 pages, 24 figure