981 research outputs found
A Survey on Neural Network Interpretability
Along with the great success of deep neural networks, there is also growing
concern about their black-box nature. The interpretability issue affects
people's trust on deep learning systems. It is also related to many ethical
problems, e.g., algorithmic discrimination. Moreover, interpretability is a
desired property for deep networks to become powerful tools in other research
fields, e.g., drug discovery and genomics. In this survey, we conduct a
comprehensive review of the neural network interpretability research. We first
clarify the definition of interpretability as it has been used in many
different contexts. Then we elaborate on the importance of interpretability and
propose a novel taxonomy organized along three dimensions: type of engagement
(passive vs. active interpretation approaches), the type of explanation, and
the focus (from local to global interpretability). This taxonomy provides a
meaningful 3D view of distribution of papers from the relevant literature as
two of the dimensions are not simply categorical but allow ordinal
subcategories. Finally, we summarize the existing interpretability evaluation
methods and suggest possible research directions inspired by our new taxonomy.Comment: This work has been accepted by IEEE-TETC
A Survey of Neural Trees
Neural networks (NNs) and decision trees (DTs) are both popular models of
machine learning, yet coming with mutually exclusive advantages and
limitations. To bring the best of the two worlds, a variety of approaches are
proposed to integrate NNs and DTs explicitly or implicitly. In this survey,
these approaches are organized in a school which we term as neural trees (NTs).
This survey aims to present a comprehensive review of NTs and attempts to
identify how they enhance the model interpretability. We first propose a
thorough taxonomy of NTs that expresses the gradual integration and
co-evolution of NNs and DTs. Afterward, we analyze NTs in terms of their
interpretability and performance, and suggest possible solutions to the
remaining challenges. Finally, this survey concludes with a discussion about
other considerations like conditional computation and promising directions
towards this field. A list of papers reviewed in this survey, along with their
corresponding codes, is available at:
https://github.com/zju-vipa/awesome-neural-treesComment: 35 pages, 7 figures and 1 tabl
Interpretable Prototype-based Graph Information Bottleneck
The success of Graph Neural Networks (GNNs) has led to a need for
understanding their decision-making process and providing explanations for
their predictions, which has given rise to explainable AI (XAI) that offers
transparent explanations for black-box models. Recently, the use of prototypes
has successfully improved the explainability of models by learning prototypes
to imply training graphs that affect the prediction. However, these approaches
tend to provide prototypes with excessive information from the entire graph,
leading to the exclusion of key substructures or the inclusion of irrelevant
substructures, which can limit both the interpretability and the performance of
the model in downstream tasks. In this work, we propose a novel framework of
explainable GNNs, called interpretable Prototype-based Graph Information
Bottleneck (PGIB) that incorporates prototype learning within the information
bottleneck framework to provide prototypes with the key subgraph from the input
graph that is important for the model prediction. This is the first work that
incorporates prototype learning into the process of identifying the key
subgraphs that have a critical impact on the prediction performance. Extensive
experiments, including qualitative analysis, demonstrate that PGIB outperforms
state-of-the-art methods in terms of both prediction performance and
explainability.Comment: NeurIPS 202
From Anecdotal Evidence to Quantitative Evaluation Methods:A Systematic Review on Evaluating Explainable AI
The rising popularity of explainable artificial intelligence (XAI) to understand high-performing black boxes raised the question of how to evaluate explanations of machine learning (ML) models. While interpretability and explainability are often presented as a subjectively validated binary property, we consider it a multi-faceted concept. We identify 12 conceptual properties, such as Compactness and Correctness, that should be evaluated for comprehensively assessing the quality of an explanation. Our so-called Co-12 properties serve as categorization scheme for systematically reviewing the evaluation practices of more than 300 papers published in the past 7 years at major AI and ML conferences that introduce an XAI method. We find that one in three papers evaluate exclusively with anecdotal evidence, and one in five papers evaluate with users. This survey also contributes to the call for objective, quantifiable evaluation methods by presenting an extensive overview of quantitative XAI evaluation methods. Our systematic collection of evaluation methods provides researchers and practitioners with concrete tools to thoroughly validate, benchmark, and compare new and existing XAI methods. The Co-12 categorization scheme and our identified evaluation methods open up opportunities to include quantitative metrics as optimization criteria during model training to optimize for accuracy and interpretability simultaneously.</p
- …