1,995 research outputs found
Learning Support and Trivial Prototypes for Interpretable Image Classification
Prototypical part network (ProtoPNet) methods have been designed to achieve
interpretable classification by associating predictions with a set of training
prototypes, which we refer to as trivial prototypes because they are trained to
lie far from the classification boundary in the feature space. Note that it is
possible to make an analogy between ProtoPNet and support vector machine (SVM)
given that the classification from both methods relies on computing similarity
with a set of training points (i.e., trivial prototypes in ProtoPNet, and
support vectors in SVM). However, while trivial prototypes are located far from
the classification boundary, support vectors are located close to this
boundary, and we argue that this discrepancy with the well-established SVM
theory can result in ProtoPNet models with inferior classification accuracy. In
this paper, we aim to improve the classification of ProtoPNet with a new method
to learn support prototypes that lie near the classification boundary in the
feature space, as suggested by the SVM theory. In addition, we target the
improvement of classification results with a new model, named ST-ProtoPNet,
which exploits our support prototypes and the trivial prototypes to provide
more effective classification. Experimental results on CUB-200-2011, Stanford
Cars, and Stanford Dogs datasets demonstrate that ST-ProtoPNet achieves
state-of-the-art classification accuracy and interpretability results. We also
show that the proposed support prototypes tend to be better localised in the
object of interest rather than in the background region
Teaching Categories to Human Learners with Visual Explanations
We study the problem of computer-assisted teaching with explanations.
Conventional approaches for machine teaching typically only provide feedback at
the instance level e.g., the category or label of the instance. However, it is
intuitive that clear explanations from a knowledgeable teacher can
significantly improve a student's ability to learn a new concept. To address
these existing limitations, we propose a teaching framework that provides
interpretable explanations as feedback and models how the learner incorporates
this additional information. In the case of images, we show that we can
automatically generate explanations that highlight the parts of the image that
are responsible for the class label. Experiments on human learners illustrate
that, on average, participants achieve better test set performance on
challenging categorization tasks when taught with our interpretable approach
compared to existing methods
Precise Proximal Femur Fracture Classification for Interactive Training and Surgical Planning
We demonstrate the feasibility of a fully automatic computer-aided diagnosis
(CAD) tool, based on deep learning, that localizes and classifies proximal
femur fractures on X-ray images according to the AO classification. The
proposed framework aims to improve patient treatment planning and provide
support for the training of trauma surgeon residents. A database of 1347
clinical radiographic studies was collected. Radiologists and trauma surgeons
annotated all fractures with bounding boxes, and provided a classification
according to the AO standard. The proposed CAD tool for the classification of
radiographs into types "A", "B" and "not-fractured", reaches a F1-score of 87%
and AUC of 0.95, when classifying fractures versus not-fractured cases it
improves up to 94% and 0.98. Prior localization of the fracture results in an
improvement with respect to full image classification. 100% of the predicted
centers of the region of interest are contained in the manually provided
bounding boxes. The system retrieves on average 9 relevant images (from the
same class) out of 10 cases. Our CAD scheme localizes, detects and further
classifies proximal femur fractures achieving results comparable to
expert-level and state-of-the-art performance. Our auxiliary localization model
was highly accurate predicting the region of interest in the radiograph. We
further investigated several strategies of verification for its adoption into
the daily clinical routine. A sensitivity analysis of the size of the ROI and
image retrieval as a clinical use case were presented.Comment: Accepted at IPCAI 2020 and IJCAR
Concept-Centric Transformers: Enhancing Model Interpretability through Object-Centric Concept Learning within a Shared Global Workspace
To explain "black-box" properties of AI models, many approaches, such as post
hoc and intrinsically interpretable models, have been proposed to provide
plausible explanations that identify human-understandable features/concepts
that a trained model uses to make predictions, and attention mechanisms have
been widely used to aid in model interpretability by visualizing that
information. However, the problem of configuring an interpretable model that
effectively communicates and coordinates among computational modules has
received less attention. A recently proposed shared global workspace theory
demonstrated that networks of distributed modules can benefit from sharing
information with a bandwidth-limited working memory because the communication
constraints encourage specialization, compositionality, and synchronization
among the modules. Inspired by this, we consider how such shared working
memories can be realized to build intrinsically interpretable models with
better interpretability and performance. Toward this end, we propose
Concept-Centric Transformers, a simple yet effective configuration of the
shared global workspace for interpretability consisting of: i) an
object-centric-based architecture for extracting semantic concepts from input
features, ii) a cross-attention mechanism between the learned concept and input
embeddings, and iii) standard classification and additional explanation losses
to allow human analysts to directly assess an explanation for the model's
classification reasoning. We test our approach against other existing
concept-based methods on classification tasks for various datasets, including
CIFAR100 (super-classes), CUB-200-2011 (bird species), and ImageNet, and we
show that our model achieves better classification accuracy than all selected
methods across all problems but also generates more consistent concept-based
explanations of classification output.Comment: 21 pages, 9 tables, 13 figure
- …