2 research outputs found
A Programmatic and Semantic Approach to Explaining and DebuggingNeural Network Based Object Detectors
Even as deep neural networks have become very effective for tasks in vision
and perception, it remains difficult to explain and debug their behavior. In
this paper, we present a programmatic and semantic approach to explaining,
understanding, and debugging the correct and incorrect behaviors of a neural
network-based perception system. Our approach is semantic in that it employs a
high-level representation of the distribution of environment scenarios that the
detector is intended to work on. It is programmatic in that scenario
representation is a program in a domain-specific probabilistic programming
language which can be used to generate synthetic data to test a given
perception module. Our framework assesses the performance of a perception
module to identify correct and incorrect detections, extracts rules from those
results that semantically characterizes the correct and incorrect scenarios,
and then specializes the probabilistic program with those rules in order to
more precisely characterize the scenarios in which the perception module
operates correctly or not. We demonstrate our results using the SCENIC
probabilistic programming language and a neural network-based object detector.
Our experiments show that it is possible to automatically generate compact
rules that significantly increase the correct detection rate (or conversely the
incorrect detection rate) of the network and can thus help with understanding
and debugging its behavior
From Anecdotal Evidence to Quantitative Evaluation Methods:A Systematic Review on Evaluating Explainable AI
The rising popularity of explainable artificial intelligence (XAI) to
understand high-performing black boxes, also raised the question of how to
evaluate explanations of machine learning (ML) models. While interpretability
and explainability are often presented as a subjectively validated binary
property, we consider it a multi-faceted concept. We identify 12 conceptual
properties, such as Compactness and Correctness, that should be evaluated for
comprehensively assessing the quality of an explanation. Our so-called Co-12
properties serve as categorization scheme for systematically reviewing the
evaluation practice of more than 300 papers published in the last 7 years at
major AI and ML conferences that introduce an XAI method. We find that 1 in 3
papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate
with users. We also contribute to the call for objective, quantifiable
evaluation methods by presenting an extensive overview of quantitative XAI
evaluation methods. This systematic collection of evaluation methods provides
researchers and practitioners with concrete tools to thoroughly validate,
benchmark and compare new and existing XAI methods. This also opens up
opportunities to include quantitative metrics as optimization criteria during
model training in order to optimize for accuracy and interpretability
simultaneously.Comment: Link to website added: https://utwente-dmb.github.io/xai-papers