491 research outputs found
From Anecdotal Evidence to Quantitative Evaluation Methods:A Systematic Review on Evaluating Explainable AI
The rising popularity of explainable artificial intelligence (XAI) to
understand high-performing black boxes, also raised the question of how to
evaluate explanations of machine learning (ML) models. While interpretability
and explainability are often presented as a subjectively validated binary
property, we consider it a multi-faceted concept. We identify 12 conceptual
properties, such as Compactness and Correctness, that should be evaluated for
comprehensively assessing the quality of an explanation. Our so-called Co-12
properties serve as categorization scheme for systematically reviewing the
evaluation practice of more than 300 papers published in the last 7 years at
major AI and ML conferences that introduce an XAI method. We find that 1 in 3
papers evaluate exclusively with anecdotal evidence, and 1 in 5 papers evaluate
with users. We also contribute to the call for objective, quantifiable
evaluation methods by presenting an extensive overview of quantitative XAI
evaluation methods. This systematic collection of evaluation methods provides
researchers and practitioners with concrete tools to thoroughly validate,
benchmark and compare new and existing XAI methods. This also opens up
opportunities to include quantitative metrics as optimization criteria during
model training in order to optimize for accuracy and interpretability
simultaneously.Comment: Link to website added: https://utwente-dmb.github.io/xai-papers
Machine Unlearning: A Survey
Machine learning has attracted widespread attention and evolved into an
enabling technology for a wide range of highly successful applications, such as
intelligent computer vision, speech recognition, medical diagnosis, and more.
Yet a special need has arisen where, due to privacy, usability, and/or the
right to be forgotten, information about some specific samples needs to be
removed from a model, called machine unlearning. This emerging technology has
drawn significant interest from both academics and industry due to its
innovation and practicality. At the same time, this ambitious problem has led
to numerous research efforts aimed at confronting its challenges. To the best
of our knowledge, no study has analyzed this complex topic or compared the
feasibility of existing unlearning solutions in different kinds of scenarios.
Accordingly, with this survey, we aim to capture the key concepts of unlearning
techniques. The existing solutions are classified and summarized based on their
characteristics within an up-to-date and comprehensive review of each
category's advantages and limitations. The survey concludes by highlighting
some of the outstanding issues with unlearning techniques, along with some
feasible directions for new research opportunities
Towards Control-Centric Representations in Reinforcement Learning from Images
Image-based Reinforcement Learning is a practical yet challenging task. A
major hurdle lies in extracting control-centric representations while
disregarding irrelevant information. While approaches that follow the
bisimulation principle exhibit the potential in learning state representations
to address this issue, they still grapple with the limited expressive capacity
of latent dynamics and the inadaptability to sparse reward environments. To
address these limitations, we introduce ReBis, which aims to capture
control-centric information by integrating reward-free control information
alongside reward-specific knowledge. ReBis utilizes a transformer architecture
to implicitly model the dynamics and incorporates block-wise masking to
eliminate spatiotemporal redundancy. Moreover, ReBis combines
bisimulation-based loss with asymmetric reconstruction loss to prevent feature
collapse in environments with sparse rewards. Empirical studies on two large
benchmarks, including Atari games and DeepMind Control Suit, demonstrate that
ReBis has superior performance compared to existing methods, proving its
effectiveness
- …