19,879 research outputs found
On Cognitive Preferences and the Plausibility of Rule-based Models
It is conventional wisdom in machine learning and data mining that logical
models such as rule sets are more interpretable than other models, and that
among such rule-based models, simpler models are more interpretable than more
complex ones. In this position paper, we question this latter assumption by
focusing on one particular aspect of interpretability, namely the plausibility
of models. Roughly speaking, we equate the plausibility of a model with the
likeliness that a user accepts it as an explanation for a prediction. In
particular, we argue that, all other things being equal, longer explanations
may be more convincing than shorter ones, and that the predominant bias for
shorter models, which is typically necessary for learning powerful
discriminative models, may not be suitable when it comes to user acceptance of
the learned models. To that end, we first recapitulate evidence for and against
this postulate, and then report the results of an evaluation in a
crowd-sourcing study based on about 3.000 judgments. The results do not reveal
a strong preference for simple rules, whereas we can observe a weak preference
for longer rules in some domains. We then relate these results to well-known
cognitive biases such as the conjunction fallacy, the representative heuristic,
or the recogition heuristic, and investigate their relation to rule length and
plausibility.Comment: V4: Another rewrite of section on interpretability to clarify focus
on plausibility and relation to interpretability, comprehensibility, and
justifiabilit
The Pragmatic Turn in Explainable Artificial Intelligence (XAI)
In this paper I argue that the search for explainable models and interpretable decisions in AI must be reformulated in terms of the broader project of offering a pragmatic and naturalistic account of understanding in AI. Intuitively, the purpose of providing an explanation of a model or a decision is to make it understandable to its stakeholders. But without a previous grasp of what it means to say that an agent understands a model or a decision, the explanatory strategies will lack a well-defined goal. Aside from providing a clearer objective for XAI, focusing on understanding also allows us to relax the factivity condition on explanation, which is impossible to fulfill in many machine learning models, and to focus instead on the pragmatic conditions that determine the best fit between a model and the methods and devices deployed to understand it. After an examination of the different types of understanding discussed in the philosophical and psychological literature, I conclude that interpretative or approximation models not only provide the best way to achieve the objectual understanding of a machine learning model, but are also a necessary condition to achieve post hoc interpretability. This conclusion is partly based on the shortcomings of the purely functionalist approach to post hoc interpretability that seems to be predominant in most recent literature
Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities
Recent advancements in AI applications to healthcare have shown incredible
promise in surpassing human performance in diagnosis and disease prognosis.
With the increasing complexity of AI models, however, concerns regarding their
opacity, potential biases, and the need for interpretability. To ensure trust
and reliability in AI systems, especially in clinical risk prediction models,
explainability becomes crucial. Explainability is usually referred to as an AI
system's ability to provide a robust interpretation of its decision-making
logic or the decisions themselves to human stakeholders. In clinical risk
prediction, other aspects of explainability like fairness, bias, trust, and
transparency also represent important concepts beyond just interpretability. In
this review, we address the relationship between these concepts as they are
often used together or interchangeably. This review also discusses recent
progress in developing explainable models for clinical risk prediction,
highlighting the importance of quantitative and clinical evaluation and
validation across multiple common modalities in clinical practice. It
emphasizes the need for external validation and the combination of diverse
interpretability methods to enhance trust and fairness. Adopting rigorous
testing, such as using synthetic datasets with known generative factors, can
further improve the reliability of explainability methods. Open access and
code-sharing resources are essential for transparency and reproducibility,
enabling the growth and trustworthiness of explainable research. While
challenges exist, an end-to-end approach to explainability in clinical risk
prediction, incorporating stakeholders from clinicians to developers, is
essential for success
- …