782 research outputs found
Interpreting Vision and Language Generative Models with Semantic Visual Priors
When applied to Image-to-text models, interpretability methods often provide
token-by-token explanations namely, they compute a visual explanation for each
token of the generated sequence. Those explanations are expensive to compute
and unable to comprehensively explain the model's output. Therefore, these
models often require some sort of approximation that eventually leads to
misleading explanations. We develop a framework based on SHAP, that allows for
generating comprehensive, meaningful explanations leveraging the meaning
representation of the output sequence as a whole. Moreover, by exploiting
semantic priors in the visual backbone, we extract an arbitrary number of
features that allows the efficient computation of Shapley values on large-scale
models, generating at the same time highly meaningful visual explanations. We
demonstrate that our method generates semantically more expressive explanations
than traditional methods at a lower compute cost and that it can be generalized
over other explainability methods
Causality-Inspired Taxonomy for Explainable Artificial Intelligence
As two sides of the same coin, causality and explainable artificial
intelligence (xAI) were initially proposed and developed with different goals.
However, the latter can only be complete when seen through the lens of the
causality framework. As such, we propose a novel causality-inspired framework
for xAI that creates an environment for the development of xAI approaches. To
show its applicability, biometrics was used as case study. For this, we have
analysed 81 research papers on a myriad of biometric modalities and different
tasks. We have categorised each of these methods according to our novel xAI
Ladder and discussed the future directions of the field
Interpreting vision and language generative models with semantic visual priors
When applied to Image-to-text models, explainability methods have two challenges. First, they often provide token-by-token explanations namely, they compute a visual explanation for each token of the generated sequence. This makes explanations expensive to compute and unable to comprehensively explain the model's output. Second, for models with visual inputs, explainability methods such as SHAP typically consider superpixels as features. Since superpixels do not correspond to semantically meaningful regions of an image, this makes explanations harder to interpret. We develop a framework based on SHAP, that allows for generating comprehensive, meaningful explanations leveraging the meaning representation of the output sequence as a whole. Moreover, by exploiting semantic priors in the visual backbone, we extract an arbitrary number of features that allow the efficient computation of Shapley values on large-scale models, generating at the same time highly meaningful visual explanations. We demonstrate that our method generates semantically more expressive explanations than traditional methods at a lower compute cost and that it can be generalized to a large family of vision-language models
Rational Shapley Values
Explaining the predictions of opaque machine learning algorithms is an
important and challenging task, especially as complex models are increasingly
used to assist in high-stakes decisions such as those arising in healthcare and
finance. Most popular tools for post-hoc explainable artificial intelligence
(XAI) are either insensitive to context (e.g., feature attributions) or
difficult to summarize (e.g., counterfactuals). In this paper, I introduce
\emph{rational Shapley values}, a novel XAI method that synthesizes and extends
these seemingly incompatible approaches in a rigorous, flexible manner. I
leverage tools from decision theory and causal modeling to formalize and
implement a pragmatic approach that resolves a number of known challenges in
XAI. By pairing the distribution of random variables with the appropriate
reference class for a given explanation task, I illustrate through theory and
experiments how user goals and knowledge can inform and constrain the solution
set in an iterative fashion. The method compares favorably to state of the art
XAI tools in a range of quantitative and qualitative comparisons.Comment: 20 pages, 3 figures, 7 table
Explainability for Large Language Models: A Survey
Large language models (LLMs) have demonstrated impressive capabilities in
natural language processing. However, their internal mechanisms are still
unclear and this lack of transparency poses unwanted risks for downstream
applications. Therefore, understanding and explaining these models is crucial
for elucidating their behaviors, limitations, and social impacts. In this
paper, we introduce a taxonomy of explainability techniques and provide a
structured overview of methods for explaining Transformer-based language
models. We categorize techniques based on the training paradigms of LLMs:
traditional fine-tuning-based paradigm and prompting-based paradigm. For each
paradigm, we summarize the goals and dominant approaches for generating local
explanations of individual predictions and global explanations of overall model
knowledge. We also discuss metrics for evaluating generated explanations, and
discuss how explanations can be leveraged to debug models and improve
performance. Lastly, we examine key challenges and emerging opportunities for
explanation techniques in the era of LLMs in comparison to conventional machine
learning models
- …