Interpretable Deep Learning: Beyond Feature-Importance with Concept-based Explanations

Deep Neural Network (DNN) models are challenging to interpret because of their highly complex and non-linear nature. This lack of interpretability (1) inhibits adoption within safety critical applications, (2) makes it challenging to debug existing models, and (3) prevents us from extracting valuable knowledge. Explainable AI (XAI) research aims to increase the transparency of DNN model behaviour to improve interpretability. Feature importance explanations are the most popular interpretability approaches. They show the importance of each input feature (e.g., pixel, patch, word vector) to the model’s prediction. However, we hypothesise that feature importance explanations have two main shortcomings concerning their inability to describe the complexity of a DNN behaviour with sufficient (1) fidelity and (2) richness. Fidelity and richness are essential because different tasks, users, and data types require specific levels of trust and understanding. The goal of this thesis is to showcase the shortcomings of feature importance explanations and to develop explanation techniques that describe the DNN behaviour with greater richness. We design an adversarial explanation attack to highlight the infidelity and inadequacy of feature importance explanations. Our attack modifies the parameters of a pre-trained model. It uses fairness as a proxy measure for the fidelity of an explanation method to demonstrate that the apparent importance of a feature does not reveal anything reliable about the fairness of a model. Hence, regulators or auditors should not rely on feature importance explanations to measure or enforce standards of fairness. As one solution, we formulate five different levels of the semantic richness of explanations to evaluate explanations and propose two function decomposition frameworks (DGINN and CME) to extract explanations from DNNs at a semantically higher level than feature importance explanations. Concept-based approaches provide explanations in terms of atomic human-understandable units (e.g., wheel or door) rather than individual raw features (e.g., pixels or characters). Our function decomposition frameworks can extract specific class representations from 5% of the network parameters and concept representations with an average-per-concept F1 score of 86%. Finally, the CME framework makes it possible to compare concept-based explanations, contributing to the scientific rigour of evaluating interpretability methods.The author would like to appreciate the generous sponsorship of the Engineering and Physical Sciences Research Council (EPSRC), The Department of Computer Science and Technology at the University of Cambridge, and Tenyks, Inc

ESTABLISHING APPROPRIATE PARAMETARS FOR ROOTING OF MICROPROPAGATED PEAR ROOTSTOCK OHF X 333 (PYRUS COMMUNIS L.)

The study was carried out in period September 2016 – April 2017, in laboratory for in vitro propagation of Fruit Growing Institute – Plovdiv. These rootstocks were obtained by hybridization of pear varieties of Old Home X Farmigdale in the American state of Oregon. Micropropagated plants in vitro were rooted on B (Dimanov, 1987) nutrient media in two different forms (solid and liquid with perlite) with different concentration of plants grow up regulators- auxins (IAA and IBA). The results show that the micropropagated plants of this rootstock are highly dependent from the nutrient media composition and concentration of grow up regulators for rooting. Plants with roots was produced 15 – 20 days after cultivated on nutrient media for rooting, but results were reported 30 days after cultivation date . According from forms (solid and liquid) and plants regulators –auxins (IBA and IAA), percent of the rooted plants it is different

Salivary duct carcinoma of the parotid gland: a review article

Background: Tumors of the parotid gland are a common pathology in the maxillofacial region. Most of them are benign. Some histological variants of the malignant ones, such as salivary duct carcinoma (SDC), are rare but have an extremely poor prognosis if not detected and treated early.Materials and methods: This is a review article written after analyzing information on the subject of malignant neoplasms of the parotid gland with the aim of providing a useful and systematic read, giving comprehensive information on the diagnosis, treatment, follow-up, and prognosis of these patients. It is based on the analysis of 44 medical articles (43 in English and 1 in Bulgarian).Discussion: SDC accounts for 1% to 3% of all malignant salivary gland tumors. Its etiology is currently unknown. It is characterized by rapid growth, pain, facial paralysis, and nodal metastases in a significant proportion of patients. Perineural spread is its hallmark. Clinical symptoms such as facial nerve palsy, trismus, dysphagia or foreign body sensation in the oropharynx, and ear pain are poor prognostic signs. The basis of the treatment of SDC is its early detection, its surgical removal within healthy limits, and semi-sedentary chemotherapy and radiation therapy in more advanced cases. Fine-needle aspiration biopsy and imaging studies such as computed tomography, nuclear magnetic resonance, and positron emission tomography, as well as its patho-anatomical verification, are key to its diagnosis. A complete sialadenectomy of the parotid gland as a treatment option involves the removal of the facial nerve when it is involved in the process, which leads to important psycho-emotional consequences for patients.Conclusion: Knowledge of the diagnosis, treatment, and prognosis of SDC is crucial in the management of this rare but extremely poor-prognosis parotid malignancy

You shouldn’t trust me: Learning models which conceal unfairness from multiple explanation methods.

Transparency of algorithmic systems is an important area of research, which has been discussed as a way for end-users and regulators to develop appropriate trust in machine learning models. One popular approach, LIME [23], even suggests that model expla- nations can answer the question “Why should I trust you?”. Here we show a straightforward method for modifying a pre-trained model to manipulate the output of many popular feature importance explana- tion methods with little change in accuracy, thus demonstrating the danger of trusting such explanation methods. We show how this ex- planation attack can mask a model’s discriminatory use of a sensitive feature, raising strong concerns about using such explanation meth- ods to check fairness of a model

Efficient Multi-Objective NeuroEvolution in Computer Vision and Applications for Threat Identification

Concealed threat detection is at the heart of critical security systems designed to en- sure public safety. Currently, methods for threat identification and detection are primarily manual, but there is a recent vision to automate the process. Problematically, developing computer vision models capable of operating in a wide range of settings, such as the ones arising in threat detection, is a challenging task involving multiple (and often conflicting) objectives. Automated machine learning (AutoML) is a flourishing field which endeavours to dis- cover and optimise models and hyperparameters autonomously, providing an alternative to classic, effort-intensive hyperparameter search. However, existing approaches typ- ically show significant downsides, like their (1) high computational cost/greediness in resources, (2) limited (or absent) scalability to custom datasets, (3) inability to provide competitive alternatives to expert-designed and heuristic approaches and (4) common consideration of a single objective. Moreover, most existing studies focus on standard classification tasks and thus cannot address a plethora of problems in threat detection and, more broadly, in a wide variety of compelling computer vision scenarios. This thesis leverages state-of-the-art convolutional autoencoders and semantic seg- mentation (Chapter 2) to develop effective multi-objective AutoML strategies for neural architecture search. These strategies are designed for threat detection and provide in- sights into some quintessential computer vision problems. To this end, the thesis first introduces two new models, a practical Multi-Objective Neuroevolutionary approach for Convolutional Autoencoders (MONCAE, Chapter 3) and a Resource-Aware model for Multi-Objective Semantic Segmentation (RAMOSS, Chapter 4). Interestingly, these ap- proaches reached state-of-the-art results using a fraction of computational resources re- quired by competing systems (0.33 GPU days compared to 3150), yet allowing for mul- tiple objectives (e.g., performance and number of parameters) to be simultaneously op- timised. This drastic speed-up was possible through the coalescence of neuroevolution algorithms with a new heuristic technique termed Progressive Stratified Sampling. The presented methods are evaluated on a range of benchmark datasets and then applied to several threat detection problems, outperforming previous attempts in balancing multiple objectives. The final chapter of the thesis focuses on thread detection, exploiting these two mod- els and novel components. It presents first a new modification of specialised proxy scores to be embedded in RAMOSS, enabling us to further accelerate the AutoML process even more drastically while maintaining avant-garde performance (above 85% precision for SIXray). This approach rendered a new automatic evolutionary Multi-objEctive method for cOncealed Weapon detection (MEOW), which outperforms state-of-the-art models for threat detection in key datasets: a gold standard benchmark (SixRay) and a security- critical, proprietary dataset. Finally, the thesis shifts the focus from neural architecture search to identifying the most representative data samples. Specifically, the Multi-objectIve Core-set Discovery through evolutionAry algorithMs in computEr vision approach (MIRA-ME) showcases how the new neural architecture search techniques developed in previous chapters can be adapted to operate on data space. MIRA-ME offers supervised and unsupervised ways to select maximally informative, compact sets of images via dataset compression. This operation can offset the computational cost further (above 90% compression), with a minimal sacrifice in performance (less than 5% for MNIST and less than 13% for SIXray). Overall, this thesis proposes novel model- and data-centred approaches towards a more widespread use of AutoML as an optimal tool for architecture and coreset discov- ery. With the presented and future developments, the work suggests that AutoML can effectively operate in real-time and performance-critical settings such as in threat de- tection, even fostering interpretability by uncovering more parsimonious optimal models. More widely, these approaches have the potential to provide effective solutions to chal- lenging computer vision problems that nowadays are typically considered unfeasible for AutoML settings

Multi-scale viscoplastic behaviour of Halite: In-situ SEM full field measurements, a micro-mechanical approach

Halite geological formations are already extensively used for underground storage of hydrocarbons. For example, the entire USA federal reserve of petrol resides in deep (500 - 1000 m) artificial salt caverns, which are realized by controlled dissolution. In France, many such salt caverns are used for storage of natural gas by GDF. Salt caverns and carries are also intended to become nuclear waste repositories. At this point, salt caverns are also seriously envisaged for the daily storage of energy from renewable, but intermittent sources (photovoltaic, Aeolian), under the form of compressed air. Halite mechanical behaviour was extensively studied for the purpose of safe geothechnical applications. Halite is a ductile type rock. Its differed (time-dependent) mechanical response dominates by far, and therefore deep salt caverns experience convergence (closure), which may result in catastrophic subsidence of the overlaying geological layers. Hence, a particular attention was drawn to characterize salt single crystal creep properties (active slip systems and critical resolved shear stresses), and the rheology of poly-crystalline salt, at various temperatures, pressures, differential stresses and water contents (Ter Heege et al., 2007). But, most studies were concerned with macroscopically derived flow laws, corresponding to rather high differential stresses (as compared with those experienced on site), where crystal slip plasticity (CSP) dominates. But, many studies have also shown that halite is very sensitive to solution-precipitation creep (SPC) mechanisms, which may result in solution transfer accommodated grain boundary sliding (GBS). Conversely, some recent studies report that halite is able to flow at ambient conditions, and under very small loads, with strain rates much faster (four orders of magnitude) than those extrapolated from high stress experiments (Bérest et al., 2005). Though, the specific creep micro-mechanisms were not identified, Bérest et al. (2005) invoked possible SPC. Additionally, the effects on long term behaviour of cyclic loading (fatigue) are still poorly known. It is therefore still questionable weather it is really possible to safely extrapolate the laboratory data to the long term envisaged geotechnical applications. To answer we need i) additional experimental work in order to up date the deformation mechanism maps on the basis of better identified micro-physical mechanisms and quantification of their respective activity; and ii) numerical modelling at the scales of the material, and of the underground storage structures, in respect with the appropriated thermo-hygro-mechaniclal loadings. In the present work, we present our preliminary investigation of viscoplastic global and local responses of synthetic fine grained (50 - 500 m) halite by the means of full field measurements (FFM) of local strain by digital image correlation (DIC) during simple compression in-situ SEM (Doumalain et al., 2003). Figure 1 shows a typical loading curve obtained incrementally at the constant strain rate of c.a. 5x10-5 s-1. CSP evidenced by the development of slip lines on the free grain surfaces, and characterized by quasi-linear strain hardening, dominates the overall response up to several % of strain (microfracturing did not appear before 8 % strain). Yet, at the scale of the microstructure, the development of viscoplastic strain is heterogeneous, as shown by the strain maps obtained by DIC and corresponding to four incremental stages of the loading sequence. The heterogeneity of the strain field relates to the loading boundary conditions and to the local microstructure, such crystal size and orientation (which is characterized by electron back scattering diffraction, EBSD). Such micromechanical approach aims to provide the basis for the development of FE (finite element) computational CSP of polycrystalline halite

Liens entre comportement multiéchelle et mécanismes locaux de la déformation à haute température et pression de silicates biphasés représentatifs de la croûte terrestre inférieure

Le comportement rhéologique à haute température d’agrégats silicatés bi-phasés a été étudié expérimentalement (essais triaxiaux et torsion à haute température et pression). Des observations au microscope à transmission et à balayage ont mis en évidence des microstructures liées à l’histoire locale de déformation. Par des calculs aux éléments finis à l’échelle de quelques grains nous cherchons à comprendre et valider la séquence de mécanismes actifs et leurs liens avec le comportement global

Now you see me (CME): Concept-based model extraction

Deep Neural Networks (DNNs) have achieved remarkable performance on a range of tasks. A key step to further empowering DNN-based approaches is improving their explainability. In this work we present CME: a concept-based model extraction framework, used for analysing DNN models via concept-based extracted models. Using two case studies (dSprites, and Caltech UCSD Birds), we demonstrate how CME can be used to (i) analyse the concept information learned by a DNN model (ii) analyse how a DNN uses this concept information when predicting output labels (iii) identify key concept information that can further improve DNN predictive performance (for one of the case studies, we showed how model accuracy can be improved by over 14%, using only 30% of the available concepts)

Full field investigation of salt deformation at room temperature: cooperation of crystal plasticity and grain sliding

International audienceWe observed with optical and scanning electron microscopy halite samples during uniaxial compression. Surface displacement fields were retrieved from digital images taken at different loading stages thanks to digital image correlation (DIC) techniques, on the basis of which we could 1) compute global and local strain fields, 2) identify two co-operational deformation mechanisms. The latter were 1) crystal slip plasticity (CSP), as evidenced by the occurrence of slip lines and computed discrete intracrystalline slip bands at the grain surfaces, 2) interfacial micro-cracking and grain boundary sliding (GBS), as evidenced by the computed relative interfacial displacements. The heterogeneities of the strain fields at the aggregate and at the grain scale, and the local contributions of each mechanism were clearly related to the microstructure, i.e. the relative crystallographic orientations of neighboring grains and the interfacial orientations with respect to the principal stress

AMELOBLASTOMA OF THE JAW BONES: CLINICAL STUDY AND CASE REPORT

Ameloblastoma's origin is the epithelium of ectodermal origin, which means they are tumors arising from the cells around the tooth root or, in close approximation, derived from the ectoderm. It is a benign but locally aggressive tumor with a high tendency to recur. Patients after ameloblastoma treatment need a life-long follow-up. We present an 82-year-old female patient diagnosed with ameloblastoma and treated by us. She has been referred to the clinic by doctor of dental medicine because of a routine panoramic rentgenography, which displayed rentgenographic evidence for a cyst-like tumor formation on her mandible. The patient did not have any complaints. The patient was reffered for further paraclinical imaging tests – dental panoramic radiography (OPG) and cone beam computed tomography (CBCT). The location and borders of the lesion were determined - it was circumferentially attached to the root of 44 tooth, well-outlined linea albuginea was present, the diameter of the lesion was approximately 26 mm. Bone curettage was the treatment plan - the surgical intervention in this volume was chosen as a consequence of the refusal of the patient of a partial mandibulectomy and according to her age – 82. The histopathological examination of the curettage revealed a locally infiltrative tumor process engaging the submitted bone and fibrous tissue. The final pathological diagnosis was conventional ameloblastoma with a predominantcanthomatous pattern. On the control panoramic rentgenography one year after the operation, no pathological changes in the field of the operative site were found