227 research outputs found
The Grammar of Interactive Explanatory Model Analysis
The growing need for in-depth analysis of predictive models leads to a series
of new methods for explaining their local and global properties. Which of these
methods is the best? It turns out that this is an ill-posed question. One
cannot sufficiently explain a black-box machine learning model using a single
method that gives only one perspective. Isolated explanations are prone to
misunderstanding, which inevitably leads to wrong or simplistic reasoning. This
problem is known as the Rashomon effect and refers to diverse, even
contradictory interpretations of the same phenomenon. Surprisingly, the
majority of methods developed for explainable machine learning focus on a
single aspect of the model behavior. In contrast, we showcase the problem of
explainability as an interactive and sequential analysis of a model. This paper
presents how different Explanatory Model Analysis (EMA) methods complement each
other and why it is essential to juxtapose them together. The introduced
process of Interactive EMA (IEMA) derives from the algorithmic side of
explainable machine learning and aims to embrace ideas developed in cognitive
sciences. We formalize the grammar of IEMA to describe potential human-model
dialogues. IEMA is implemented in the human-centered framework that adopts
interactivity, customizability and automation as its main traits. Combined,
these methods enhance the responsible approach to predictive modeling.Comment: 17 pages, 10 figures, 3 table
Adversarial Attacks and Defenses in Explainable Artificial Intelligence: A Survey
Explainable artificial intelligence (XAI) methods are portrayed as a remedy
for debugging and trusting statistical and deep learning models, as well as
interpreting their predictions. However, recent advances in adversarial machine
learning (AdvML) highlight the limitations and vulnerabilities of
state-of-the-art explanation methods, putting their security and
trustworthiness into question. The possibility of manipulating, fooling or
fairwashing evidence of the model's reasoning has detrimental consequences when
applied in high-stakes decision-making and knowledge discovery. This survey
provides a comprehensive overview of research concerning adversarial attacks on
explanations of machine learning models, as well as fairness metrics. We
introduce a unified notation and taxonomy of methods facilitating a common
ground for researchers and practitioners from the intersecting research fields
of AdvML and XAI. We discuss how to defend against attacks and design robust
interpretation methods. We contribute a list of existing insecurities in XAI
and outline the emerging research directions in adversarial XAI (AdvXAI).
Future work should address improving explanation methods and evaluation
protocols to take into account the reported safety issues.Comment: A shorter version of this paper was presented at the IJCAI 2023
Workshop on Explainable A
Electrical Characterization of PbZr0.4Ti0.6O3 Capacitors
We have conducted a careful study of current-voltage (I-V) characteristics in
fully integrated commercial PbZr0.4Ti0.6O3 thin film capacitors with Pt bottom
and Ir/IrO2 top electrodes. Highly reproducible steady state I-V were obtained
at various temperatures over two decades in voltage from current-time data and
analyzed in terms of several common transport models including space charge
limited conduction, Schottky thermionic emission under full and partial
depletion and Poole-Frenkel conduction, showing that the later is the most
plausible leakage mechanism in these high quality films. In addition,
ferroelectric hysteresis loops and capacitance-voltage data were obtained over
a large range of temperatures and discussed in terms of a modified
Landau-Ginzburg-Devonshire theory accounting for space charge effects.Comment: 17 pages, 7 figure
Towards Evaluating Explanations of Vision Transformers for Medical Imaging
As deep learning models increasingly find applications in critical domains
such as medical imaging, the need for transparent and trustworthy
decision-making becomes paramount. Many explainability methods provide insights
into how these models make predictions by attributing importance to input
features. As Vision Transformer (ViT) becomes a promising alternative to
convolutional neural networks for image classification, its interpretability
remains an open research question. This paper investigates the performance of
various interpretation methods on a ViT applied to classify chest X-ray images.
We introduce the notion of evaluating faithfulness, sensitivity, and complexity
of ViT explanations. The obtained results indicate that Layerwise relevance
propagation for transformers outperforms Local interpretable model-agnostic
explanations and Attention visualization, providing a more accurate and
reliable representation of what a ViT has actually learned. Our findings
provide insights into the applicability of ViT explanations in medical imaging
and highlight the importance of using appropriate evaluation criteria for
comparing them.Comment: Accepted by XAI4CV Workshop at CVPR 202
Performance is not enough: the story told by a Rashomon quartet
Predictive modelling is often reduced to finding the best model that
optimizes a selected performance measure. But what if the second-best model
describes the data in a completely different way? What about the third-best? Is
it possible that the equally effective models describe different relationships
in the data? Inspired by Anscombe's quartet, this paper introduces a Rashomon
quartet, a four models built on synthetic dataset which have practically
identical predictive performance. However, their visualization reveals distinct
explanations of the relation between input variables and the target variable.
The illustrative example aims to encourage the use of visualization to compare
predictive models beyond their performance
SurvSHAP(t): Time-dependent explanations of machine learning survival models
Machine and deep learning survival models demonstrate similar or even
improved time-to-event prediction capabilities compared to classical
statistical learning methods yet are too complex to be interpreted by humans.
Several model-agnostic explanations are available to overcome this issue;
however, none directly explain the survival function prediction. In this paper,
we introduce SurvSHAP(t), the first time-dependent explanation that allows for
interpreting survival black-box models. It is based on SHapley Additive
exPlanations with solid theoretical foundations and a broad adoption among
machine learning practitioners. The proposed methods aim to enhance precision
diagnostics and support domain experts in making decisions. Experiments on
synthetic and medical data confirm that SurvSHAP(t) can detect variables with a
time-dependent effect, and its aggregation is a better determinant of the
importance of variables for a prediction than SurvLIME. SurvSHAP(t) is
model-agnostic and can be applied to all models with functional output. We
provide an accessible implementation of time-dependent explanations in Python
at http://github.com/MI2DataLab/survshap
Observation of ion gettering effect in high temperature superconducting oxide material
Ion gettering effect has been observed in high-temperature superconducting YBa2Cu3O7 material. Silicon ions were implanted into the material and subsequent high-temperature annealing produced ion movement from a low concentration region to a higher concentration region where the damage of the crystal structure is severe. This gettering effect could be used to make a superconductor-nonsuperconductor-superconductor trilayer structure within a single YBCO film.published_or_final_versio
Hospital Length of Stay Prediction Based on Multi-modal Data towards Trustworthy Human-AI Collaboration in Radiomics
To what extent can the patient's length of stay in a hospital be predicted
using only an X-ray image? We answer this question by comparing the performance
of machine learning survival models on a novel multi-modal dataset created from
1235 images with textual radiology reports annotated by humans. Although
black-box models predict better on average than interpretable ones, like Cox
proportional hazards, they are not inherently understandable. To overcome this
trust issue, we introduce time-dependent model explanations into the human-AI
decision making process. Explaining models built on both: human-annotated and
algorithm-extracted radiomics features provides valuable insights for
physicians working in a hospital. We believe the presented approach to be
general and widely applicable to other time-to-event medical use cases. For
reproducibility, we open-source code and the TLOS dataset at
https://github.com/mi2datalab/xlungs-trustworthy-los-prediction.Comment: Accepted at International Conference on Artificial Intelligence in
Medicine (AIME 2023
Funkcje aktotwórców na przykładzie archiwów państwowych. Rozważania wstępne
The presentation by the International Council of Archives of the project of a new model of archival description "Records in Contexts" made the authors look for the possibility of using this model in the Polish system of archival information. They draw attention to the lack of studies or thesaurus in Poland containing systematized functions performed by various entities operating in all areas. They refer to a handful of examples from other countries, including the function thesaurus for The Australian Governments' Interactive Functions Thesaurus, which was one of the foundations for the preparation of the International Standard for Describing Functions ISDF and Access to Memory AtoM, and the later guidebook from the British Commonwealth – Strategies for Documenting Government Business: the DIRKS Manual. Then, they indicate the eight basic areas of activity of state archives specified in the Law on the national archival holdings and archives, i.e. shaping the state archival holdings, collecting, inventorying, storing, processing, securing and making available archival materials, and conducting information activities. Some regulations and documents prepared in the archives were subject to a preliminary partial analysis, i.e. the uniform substantive records classification scheme, the model report on the activities of the state archives and the Strategy of the State Archives for 2010–2020, as sources that can provide information when creating the Polish dictionary of functions, at least in the scope of the activities of state archives. As the theoretical model that can be used in these works, the Archival Thesaurus by Stanisław Nawrocki was pointed out. The study ends with questions about the possibility of applying in Poland an information system approach based on the analysis of functions and documentation produced during their performance, which are an incentive to discuss the shape of the system of information about the holdings.Przedstawienie przez Międzynarodową Radę Archiwów projektu nowego modelu opisu archiwalnego „Records in Contexts” sprawiło, że autorzy poszukują możliwości zastosowania tego modelu w polskim systemu informacji archiwalnej. Zwracają uwagę na brak w Polsce opracowań czy tezaurusa zawierającego usystematyzowane funkcje pełnione przez różne podmioty działające we wszelkich obszarach. Odwołują się do nielicznych przykładów z innych państw, w tym tezaurusa funkcji The Australian Governments' Interactive Functions Thesaurus, który był jedną z podstaw przygotowania Międzynarodowego standardu opisu funkcji ISDF i systemu Access to Memory AtoM, oraz późniejszego poradnika z Brytyjskiej Wspólnoty Narodów Strategies for Documenting Government Business: the DIRKS Manual. Następnie wskazują osiem określonych w ustawie o narodowym zasobie archiwalnym i archiwach podstawowych obszarów działalności archiwów państwowych, tzn. kształtowanie państwowego zasobu archiwalnego, gromadzenie, ewidencjonowanie, przechowywanie, opracowanie, zabezpieczenie i udostępnianie materiałów archiwalnych oraz prowadzenie działalności informacyjnej. Wstępnej częściowej analizie poddane zostały niektóre przepisy i dokumenty sporządzane w archiwach, tzn. jednolity rzeczowy wykaz akt (jrwa), wzór sprawozdania z działalności archiwum państwowego oraz Strategia Archiwów Państwowych na lata 2010–2020, jako źródła mogące dostarczyć informacji przy tworzeniu polskiego słownika funkcji, przynajmniej w zakresie działalności archiwów państwowych. Jako model teoretyczny, który można przy tych pracach wykorzystać, wskazany został Tezaurus archiwistyki, autorstwa Stanisława Nawrockiego. Opracowanie kończą pytania o możliwość zastosowania w Polsce podejścia do systemu informacji opartego na analizie funkcji i dokumentacji wytwarzanej w trakcie ich wykonywania, które są zachętą do podjęcia dyskusji nad kształtem systemu informacji o zasobie.
- …