Now you see me (CME): Concept-based model extraction

Deep Neural Networks (DNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering DNN-based approaches is improving their explainability. In this work we present CME: a concept-based model extraction framework, used for analysing DNN models via concept-based extracted models. Using two case studies (dSprites, and Caltech UCSD Birds), we demonstrate how CME can be used to (i) analyse the concept information learned by a DNN model (ii) analyse how a DNN uses this concept information when predicting output labels (iii) identify key concept information that can further improve DNN predictive performance (for one of the case studies, we showed how model accuracy can be improved by over 14%, using only 30% of the available concepts)

Earthquake nucleation in the lower crust by local stress amplification

Deep intracontinental earthquakes are poorly understood, despite their potential to cause significant destruction. Although lower crustal strength is currently a topic of debate, dry lower continental crust may be strong under high-grade conditions. Such strength could enable earthquake slip at high differential stress within a predominantly viscous regime, but requires further documentation in nature. Here, we analyse geological observations of seismic structures in exhumed lower crustal rocks. A granulite facies shear zone network dissects an anorthosite intrusion in Lofoten, northern Norway, and separates relatively undeformed, microcracked blocks of anorthosite. In these blocks, pristine pseudotachylytes decorate fault sets that link adjacent or intersecting shear zones. These fossil seismogenic faults are rarely >15 m in length, yet record single-event displacements of tens of centimetres, a slip/length ratio that implies >1 GPa stress drops. These pseudotachylytes represent direct identification of earthquake nucleation as a transient consequence of ongoing, localised aseismic creep

You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.

Transparency of algorithmic systems is an important area of research, which has been discussed as a way for end-users and regulators to develop appropriate trust in machine learning models. One popular approach, LIME [23], even suggests that model expla- nations can answer the question “Why should I trust you?”. Here we show a straightforward method for modifying a pre-trained model to manipulate the output of many popular feature importance explana- tion methods with little change in accuracy, thus demonstrating the danger of trusting such explanation methods. We show how this ex- planation attack can mask a model’s discriminatory use of a sensitive feature, raising strong concerns about using such explanation meth- ods to check fairness of a model

Now you see me (CME): Concept-based model extraction

Deep Neural Networks (DNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering DNN-based approaches is improving their explainability. In this work we present CME: a concept-based model extraction framework, used for analysing DNN models via concept-based extracted models. Using two case studies (dSprites, and Caltech UCSD Birds), we demonstrate how CME can be used to (i) analyse the concept information learned by a DNN model (ii) analyse how a DNN uses this concept information when predicting output labels (iii) identify key concept information that can further improve DNN predictive performance (for one of the case studies, we showed how model accuracy can be improved by over 14%, using only 30% of the available concepts)

Explainer Divergence Scores (EDS): Some Post-Hoc Explanations May be Effective for Detecting Unknown Spurious Correlations

Recent work has suggested post-hoc explainers might be ineffective for detecting spurious correlations in Deep Neural Networks (DNNs). However, we show there are serious weaknesses with the existing evaluation frameworks for this setting. Previously proposed metrics are extremely difficult to interpret and are not directly comparable between explainer methods. To alleviate these constraints, we propose a new evaluation methodology, Explainer Divergence Scores (EDS), grounded in an information theory approach to evaluate explainers. EDS is easy to interpret and naturally comparable across explainers. We use our methodology to compare the detection performance of three different explainers - feature attribution methods, influential examples and concept extraction, on two different image datasets. We discover post-hoc explainers often contain substantial information about a DNN’s dependence on spurious artifacts, but in ways often imperceptible to human users. This suggests the need for new techniques that can use this information to better detect a DNN’s reliance on spurious correlations

Is Disentanglement all you need? Comparing Concept-based & Disentanglement Approaches

Concept-based explanations have emerged as a popular way of extracting human-interpretable representations from deep discriminative models. At the same time, the disentanglement learning literature has focused on extracting similar representations in an unsupervised or weakly-supervised way, using deep generative models. Despite the overlapping goals and potential synergies, to our knowledge, there has not yet been a systematic comparison of the limitations and trade-offs between concept-based explanations and disentanglement approaches. In this paper, we give an overview of these fields, comparing and contrasting their properties and behaviours on a diverse set of tasks, and highlighting their potential strengths and limitations. In particular, we demonstrate that state-of-the-art approaches from both classes can be data inefficient, sensitive to the specific nature of the classification/regression task, or sensitive to the employed concept representation

Brittle and semibrittle creep of Tavel limestone deformed at room temperature

International audienceDeformation and failure mode of carbonate rocks depend on the confining pressure. In this study, the mechanical behavior of a limestone with an initial porosity of 14.7% is investigated at constant stress. At confining pressures below 55 MPa, dilatancy associated with microfracturing occurs during constant stress steps, ultimately leading to failure, similar to creep in other brittle media. At confining pressures higher than 55 MPa, depending on applied differential stress, inelastic compaction occurs, accommodated by crystal plasticity and characterized by constant ultrasonic wave velocities, or dilatancy resulting from nucleation and propagation of cracks due to local stress concentrations associated with dislocation pileups, ultimately causing failure. Strain rates during secondary creep preceding dilative brittle failure are sensitive to stress, while rates during compactive creep exhibit an insensitivity to stress indicative of the operation of crystal plasticity, in agreement with elastic wave velocity evolution and microstructural observations