5 research outputs found
Leveraging Rationales to Improve Human Task Performance
Machine learning (ML) systems across many application areas are increasingly
demonstrating performance that is beyond that of humans. In response to the
proliferation of such models, the field of Explainable AI (XAI) has sought to
develop techniques that enhance the transparency and interpretability of
machine learning methods. In this work, we consider a question not previously
explored within the XAI and ML communities: Given a computational system whose
performance exceeds that of its human user, can explainable AI capabilities be
leveraged to improve the performance of the human? We study this question in
the context of the game of Chess, for which computational game engines that
surpass the performance of the average player are widely available. We
introduce the Rationale-Generating Algorithm, an automated technique for
generating rationales for utility-based computational methods, which we
evaluate with a multi-day user study against two baselines. The results show
that our approach produces rationales that lead to statistically significant
improvement in human task performance, demonstrating that rationales
automatically generated from an AI's internal task model can be used not only
to explain what the system is doing, but also to instruct the user and
ultimately improve their task performance.Comment: ACM IUI 202
Decision Theoretic Foundations for Experiments Evaluating Human Decisions
Decision-making with information displays is a key focus of research in areas
like explainable AI, human-AI teaming, and data visualization. However, what
constitutes a decision problem, and what is required for an experiment to be
capable of concluding that human decisions are flawed in some way, remain open
to speculation. We present a widely applicable definition of a decision problem
synthesized from statistical decision theory and information economics. We
argue that to attribute loss in human performance to forms of bias, an
experiment must provide participants with the information that a rational agent
would need to identify the normative decision. We evaluate the extent to which
recent evaluations of decision-making from the literature on AI-assisted
decisions achieve this criteria. We find that only 10 (26\%) of 39 studies that
claim to identify biased behavior present participants with sufficient
information to characterize their behavior as deviating from good
decision-making in at least one treatment condition. We motivate the value of
studying well-defined decision problems by describing a characterization of
performance losses they allow us to conceive. In contrast, the ambiguities of a
poorly communicated decision problem preclude normative interpretation. We
conclude with recommendations for practice
Rationalization for Explainable NLP: A Survey
Recent advances in deep learning have improved the performance of many
Natural Language Processing (NLP) tasks such as translation,
question-answering, and text classification. However, this improvement comes at
the expense of model explainability. Black-box models make it difficult to
understand the internals of a system and the process it takes to arrive at an
output. Numerical (LIME, Shapley) and visualization (saliency heatmap)
explainability techniques are helpful; however, they are insufficient because
they require specialized knowledge. These factors led rationalization to emerge
as a more accessible explainable technique in NLP. Rationalization justifies a
model's output by providing a natural language explanation (rationale). Recent
improvements in natural language generation have made rationalization an
attractive technique because it is intuitive, human-comprehensible, and
accessible to non-technical users. Since rationalization is a relatively new
field, it is disorganized. As the first survey, rationalization literature in
NLP from 2007-2022 is analyzed. This survey presents available methods,
explainable evaluations, code, and datasets used across various NLP tasks that
use rationalization. Further, a new subfield in Explainable AI (XAI), namely,
Rational AI (RAI), is introduced to advance the current state of
rationalization. A discussion on observed insights, challenges, and future
directions is provided to point to promising research opportunities
Algorithmic Decision-Making Safeguarded by Human Knowledge
Commercial AI solutions provide analysts and managers with data-driven
business intelligence for a wide range of decisions, such as demand forecasting
and pricing. However, human analysts may have their own insights and
experiences about the decision-making that is at odds with the algorithmic
recommendation. In view of such a conflict, we provide a general analytical
framework to study the augmentation of algorithmic decisions with human
knowledge: the analyst uses the knowledge to set a guardrail by which the
algorithmic decision is clipped if the algorithmic output is out of bound, and
seems unreasonable. We study the conditions under which the augmentation is
beneficial relative to the raw algorithmic decision. We show that when the
algorithmic decision is asymptotically optimal with large data, the
non-data-driven human guardrail usually provides no benefit. However, we point
out three common pitfalls of the algorithmic decision: (1) lack of domain
knowledge, such as the market competition, (2) model misspecification, and (3)
data contamination. In these cases, even with sufficient data, the augmentation
from human knowledge can still improve the performance of the algorithmic
decision