The Grammar of Interactive Explanatory Model Analysis

The growing need for in-depth analysis of predictive models leads to a series of new methods for explaining their local and global properties. Which of these methods is the best? It turns out that this is an ill-posed question. One cannot sufficiently explain a black-box machine learning model using a single method that gives only one perspective. Isolated explanations are prone to misunderstanding, which inevitably leads to wrong or simplistic reasoning. This problem is known as the Rashomon effect and refers to diverse, even contradictory interpretations of the same phenomenon. Surprisingly, the majority of methods developed for explainable machine learning focus on a single aspect of the model behavior. In contrast, we showcase the problem of explainability as an interactive and sequential analysis of a model. This paper presents how different Explanatory Model Analysis (EMA) methods complement each other and why it is essential to juxtapose them together. The introduced process of Interactive EMA (IEMA) derives from the algorithmic side of explainable machine learning and aims to embrace ideas developed in cognitive sciences. We formalize the grammar of IEMA to describe potential human-model dialogues. IEMA is implemented in the human-centered framework that adopts interactivity, customizability and automation as its main traits. Combined, these methods enhance the responsible approach to predictive modeling.Comment: 17 pages, 10 figures, 3 table

Adversarial Attacks and Defenses in Explainable Artificial Intelligence: A Survey

Explainable artificial intelligence (XAI) methods are portrayed as a remedy for debugging and trusting statistical and deep learning models, as well as interpreting their predictions. However, recent advances in adversarial machine learning (AdvML) highlight the limitations and vulnerabilities of state-of-the-art explanation methods, putting their security and trustworthiness into question. The possibility of manipulating, fooling or fairwashing evidence of the model's reasoning has detrimental consequences when applied in high-stakes decision-making and knowledge discovery. This survey provides a comprehensive overview of research concerning adversarial attacks on explanations of machine learning models, as well as fairness metrics. We introduce a unified notation and taxonomy of methods facilitating a common ground for researchers and practitioners from the intersecting research fields of AdvML and XAI. We discuss how to defend against attacks and design robust interpretation methods. We contribute a list of existing insecurities in XAI and outline the emerging research directions in adversarial XAI (AdvXAI). Future work should address improving explanation methods and evaluation protocols to take into account the reported safety issues.Comment: A shorter version of this paper was presented at the IJCAI 2023 Workshop on Explainable A

Electrical Characterization of PbZr0.4Ti0.6O3 Capacitors

We have conducted a careful study of current-voltage (I-V) characteristics in fully integrated commercial PbZr0.4Ti0.6O3 thin film capacitors with Pt bottom and Ir/IrO2 top electrodes. Highly reproducible steady state I-V were obtained at various temperatures over two decades in voltage from current-time data and analyzed in terms of several common transport models including space charge limited conduction, Schottky thermionic emission under full and partial depletion and Poole-Frenkel conduction, showing that the later is the most plausible leakage mechanism in these high quality films. In addition, ferroelectric hysteresis loops and capacitance-voltage data were obtained over a large range of temperatures and discussed in terms of a modified Landau-Ginzburg-Devonshire theory accounting for space charge effects.Comment: 17 pages, 7 figure

Towards Evaluating Explanations of Vision Transformers for Medical Imaging

As deep learning models increasingly find applications in critical domains such as medical imaging, the need for transparent and trustworthy decision-making becomes paramount. Many explainability methods provide insights into how these models make predictions by attributing importance to input features. As Vision Transformer (ViT) becomes a promising alternative to convolutional neural networks for image classification, its interpretability remains an open research question. This paper investigates the performance of various interpretation methods on a ViT applied to classify chest X-ray images. We introduce the notion of evaluating faithfulness, sensitivity, and complexity of ViT explanations. The obtained results indicate that Layerwise relevance propagation for transformers outperforms Local interpretable model-agnostic explanations and Attention visualization, providing a more accurate and reliable representation of what a ViT has actually learned. Our findings provide insights into the applicability of ViT explanations in medical imaging and highlight the importance of using appropriate evaluation criteria for comparing them.Comment: Accepted by XAI4CV Workshop at CVPR 202

Performance is not enough: the story told by a Rashomon quartet

Predictive modelling is often reduced to finding the best model that optimizes a selected performance measure. But what if the second-best model describes the data in a completely different way? What about the third-best? Is it possible that the equally effective models describe different relationships in the data? Inspired by Anscombe's quartet, this paper introduces a Rashomon quartet, a four models built on synthetic dataset which have practically identical predictive performance. However, their visualization reveals distinct explanations of the relation between input variables and the target variable. The illustrative example aims to encourage the use of visualization to compare predictive models beyond their performance

SurvSHAP(t): Time-dependent explanations of machine learning survival models

Machine and deep learning survival models demonstrate similar or even improved time-to-event prediction capabilities compared to classical statistical learning methods yet are too complex to be interpreted by humans. Several model-agnostic explanations are available to overcome this issue; however, none directly explain the survival function prediction. In this paper, we introduce SurvSHAP(t), the first time-dependent explanation that allows for interpreting survival black-box models. It is based on SHapley Additive exPlanations with solid theoretical foundations and a broad adoption among machine learning practitioners. The proposed methods aim to enhance precision diagnostics and support domain experts in making decisions. Experiments on synthetic and medical data confirm that SurvSHAP(t) can detect variables with a time-dependent effect, and its aggregation is a better determinant of the importance of variables for a prediction than SurvLIME. SurvSHAP(t) is model-agnostic and can be applied to all models with functional output. We provide an accessible implementation of time-dependent explanations in Python at http://github.com/MI2DataLab/survshap

Observation of ion gettering effect in high temperature superconducting oxide material

Ion gettering effect has been observed in high-temperature superconducting YBa2Cu3O7 material. Silicon ions were implanted into the material and subsequent high-temperature annealing produced ion movement from a low concentration region to a higher concentration region where the damage of the crystal structure is severe. This gettering effect could be used to make a superconductor-nonsuperconductor-superconductor trilayer structure within a single YBCO film.published_or_final_versio

Hospital Length of Stay Prediction Based on Multi-modal Data towards Trustworthy Human-AI Collaboration in Radiomics

To what extent can the patient's length of stay in a hospital be predicted using only an X-ray image? We answer this question by comparing the performance of machine learning survival models on a novel multi-modal dataset created from 1235 images with textual radiology reports annotated by humans. Although black-box models predict better on average than interpretable ones, like Cox proportional hazards, they are not inherently understandable. To overcome this trust issue, we introduce time-dependent model explanations into the human-AI decision making process. Explaining models built on both: human-annotated and algorithm-extracted radiomics features provides valuable insights for physicians working in a hospital. We believe the presented approach to be general and widely applicable to other time-to-event medical use cases. For reproducibility, we open-source code and the TLOS dataset at https://github.com/mi2datalab/xlungs-trustworthy-los-prediction.Comment: Accepted at International Conference on Artificial Intelligence in Medicine (AIME 2023

Funkcje aktotwórców na przykładzie archiwów państwowych. Rozważania wstępne

The presentation by the International Council of Archives of the project of a new model of archival description "Records in Contexts" made the authors look for the possibility of using this model in the Polish system of archival information. They draw attention to the lack of studies or thesaurus in Poland containing systematized functions performed by various entities operating in all areas. They refer to a handful of examples from other countries, including the function thesaurus for The Australian Governments' Interactive Functions Thesaurus, which was one of the foundations for the preparation of the International Standard for Describing Functions ISDF and Access to Memory AtoM, and the later guidebook from the British Commonwealth – Strategies for Documenting Government Business: the DIRKS Manual. Then, they indicate the eight basic areas of activity of state archives specified in the Law on the national archival holdings and archives, i.e. shaping the state archival holdings, collecting, inventorying, storing, processing, securing and making available archival materials, and conducting information activities. Some regulations and documents prepared in the archives were subject to a preliminary partial analysis, i.e. the uniform substantive records classification scheme, the model report on the activities of the state archives and the Strategy of the State Archives for 2010–2020, as sources that can provide information when creating the Polish dictionary of functions, at least in the scope of the activities of state archives. As the theoretical model that can be used in these works, the Archival Thesaurus by Stanisław Nawrocki was pointed out. The study ends with questions about the possibility of applying in Poland an information system approach based on the analysis of functions and documentation produced during their performance, which are an incentive to discuss the shape of the system of information about the holdings.Przedstawienie przez Międzynarodową Radę Archiwów projektu nowego modelu opisu archiwalnego „Records in Contexts” sprawiło, że autorzy poszukują możliwości zastosowania tego modelu w polskim systemu informacji archiwalnej. Zwracają uwagę na brak w Polsce opracowań czy tezaurusa zawierającego usystematyzowane funkcje pełnione przez różne podmioty działające we wszelkich obszarach. Odwołują się do nielicznych przykładów z innych państw, w tym tezaurusa funkcji The Australian Governments' Interactive Functions Thesaurus, który był jedną z podstaw przygotowania Międzynarodowego standardu opisu funkcji ISDF i systemu Access to Memory AtoM, oraz późniejszego poradnika z Brytyjskiej Wspólnoty Narodów Strategies for Documenting Government Business: the DIRKS Manual. Następnie wskazują osiem określonych w ustawie o narodowym zasobie archiwalnym i archiwach podstawowych obszarów działalności archiwów państwowych, tzn. kształtowanie państwowego zasobu archiwalnego, gromadzenie, ewidencjonowanie, przechowywanie, opracowanie, zabezpieczenie i udostępnianie materiałów archiwalnych oraz prowadzenie działalności informacyjnej. Wstępnej częściowej analizie poddane zostały niektóre przepisy i dokumenty sporządzane w archiwach, tzn. jednolity rzeczowy wykaz akt (jrwa), wzór sprawozdania z działalności archiwum państwowego oraz Strategia Archiwów Państwowych na lata 2010–2020, jako źródła mogące dostarczyć informacji przy tworzeniu polskiego słownika funkcji, przynajmniej w zakresie działalności archiwów państwowych. Jako model teoretyczny, który można przy tych pracach wykorzystać, wskazany został Tezaurus archiwistyki, autorstwa Stanisława Nawrockiego. Opracowanie kończą pytania o możliwość zastosowania w Polsce podejścia do systemu informacji opartego na analizie funkcji i dokumentacji wytwarzanej w trakcie ich wykonywania, które są zachętą do podjęcia dyskusji nad kształtem systemu informacji o zasobie.