4 research outputs found
Explaining with Counter Visual Attributes and Examples
In this paper, we aim to explain the decisions of neural networks by
utilizing multimodal information. That is counter-intuitive attributes and
counter visual examples which appear when perturbed samples are introduced.
Different from previous work on interpreting decisions using saliency maps,
text, or visual patches we propose to use attributes and counter-attributes,
and examples and counter-examples as part of the visual explanations. When
humans explain visual decisions they tend to do so by providing attributes and
examples. Hence, inspired by the way of human explanations in this paper we
provide attribute-based and example-based explanations. Moreover, humans also
tend to explain their visual decisions by adding counter-attributes and
counter-examples to explain what is not seen. We introduce directed
perturbations in the examples to observe which attribute values change when
classifying the examples into the counter classes. This delivers intuitive
counter-attributes and counter-examples. Our experiments with both coarse and
fine-grained datasets show that attributes provide discriminating and
human-understandable intuitive and counter-intuitive explanations.Comment: arXiv admin note: substantial text overlap with arXiv:1910.07416,
arXiv:1904.0827
Multimodal Explainable Artificial Intelligence: A Comprehensive Review of Methodological Advances and Future Research Directions
The current study focuses on systematically analyzing the recent advances in
the field of Multimodal eXplainable Artificial Intelligence (MXAI). In
particular, the relevant primary prediction tasks and publicly available
datasets are initially described. Subsequently, a structured presentation of
the MXAI methods of the literature is provided, taking into account the
following criteria: a) The number of the involved modalities, b) The stage at
which explanations are produced, and c) The type of the adopted methodology
(i.e. mathematical formalism). Then, the metrics used for MXAI evaluation are
discussed. Finally, a comprehensive analysis of current challenges and future
research directions is provided.Comment: 26 pages, 11 figure