1,231 research outputs found

    Adversarial Infidelity Learning for Model Interpretation

    Full text link
    Model interpretation is essential in data mining and knowledge discovery. It can help understand the intrinsic model working mechanism and check if the model has undesired characteristics. A popular way of performing model interpretation is Instance-wise Feature Selection (IFS), which provides an importance score of each feature representing the data samples to explain how the model generates the specific output. In this paper, we propose a Model-agnostic Effective Efficient Direct (MEED) IFS framework for model interpretation, mitigating concerns about sanity, combinatorial shortcuts, model identifiability, and information transmission. Also, we focus on the following setting: using selected features to directly predict the output of the given model, which serves as a primary evaluation metric for model-interpretation methods. Apart from the features, we involve the output of the given model as an additional input to learn an explainer based on more accurate information. To learn the explainer, besides fidelity, we propose an Adversarial Infidelity Learning (AIL) mechanism to boost the explanation learning by screening relatively unimportant features. Through theoretical and experimental analysis, we show that our AIL mechanism can help learn the desired conditional distribution between selected features and targets. Moreover, we extend our framework by integrating efficient interpretation methods as proper priors to provide a warm start. Comprehensive empirical evaluation results are provided by quantitative metrics and human evaluation to demonstrate the effectiveness and superiority of our proposed method. Our code is publicly available online at https://github.com/langlrsw/MEED.Comment: 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '20), August 23--27, 2020, Virtual Event, US

    Explainable Machine Learning for Robust Modelling in Healthcare

    Get PDF
    Deep Learning (DL) has seen an unprecedented rise in popularity over the last decade, with applications ranging from machine translation to self-driving cars. This includes extensive work in sensitive domains such as healthcare and finance with, for example, models recently achieving better-than-human performance in tasks such as chest x-ray diagnosis. However, despite these impressive results there are relatively few real-world deployments of DL models in sensitive scenarios, with experts claiming this is due to a lack of model transparency, reproducibility, robustness and privacy; this is in spite of numerous techniques having been proposed to address these issues. Most notably is the development of Explainable Deep Learning techniques, which aim to compute feature importance values for a given input (i.e. which features does a model use to make its decision?) - such methods can greatly improve the transparency of a model, but have little impact on reproducibility, robustness and privacy. In this thesis, I explore how explainability techniques can be used to address these issues, by using feature attributions to improve our understanding of how model parameters change during training, and across different hyperparameter setups. Through the introduction of a novel model architecture and training technique that used model explanations to improve model consistency, I show how explanations can improve privacy, robustness and reproducibility. Extensive experimentation is carried out across a number of sensitive datasets from healthcare and bioinformatics in both traditional and federated learning settings show that these techniques have a significant impact on the quality of these models. I discuss the impact these results could have on real-world applications of deep learning, due to the issues addressed by the proposed techniques, and present some ideas for further research in this area

    Misplaced Fidelity

    Get PDF
    This paper is a review essay of W. Bradley Wendel\u27s Lawyers and Fidelity to Law, part of a symposium on Wendel\u27s book. Parts I and II aim to situate Wendel\u27s book within the literature on philosophical or theoretical legal ethics. I focus on two points: Wendel\u27s argument that legal ethics should be examined through the lens of political theory rather than moral philosophy, and his emphasis on the role law plays in setting terms of social coexistence in the midst of moral pluralism. Both of these themes lead him to reject viewing legal ethics as an instance of the problem of role morality. In part III I note the similarity between Wendel\u27s view and that of legal process theorists, and I argue that the view involves too much complacency about the American legal system. Part IV examines the central metaphor of Wendel\u27s book, fidelity to law. I distinguish between two forms of fidelity, personal and interpretive. The former is a relation between persons, while the latter means mimetic accuracy in interpretation, translation, performance of music, portraiture, or other forms of representation. I agree with Wendel\u27s views on the requirement that lawyers exhibit interpretive fidelity toward law, but not personal fidelity. I argue that law is not the kind of thing toward which one can have personal fidelity; rather, the fidelity must be toward other members of the community rather than toward norms as such; and in cases where the law systematically discriminates, or is otherwise systematically unjust, the bonds of reciprocity grounding such a relation are absent, and the kind of unconditional obedience to law that Wendel supports is unjustified. Part V asks where, on Wendel’s view, the morality went. I argue that Wendel\u27s view, which derives from but modifies Joseph Raz\u27s analysis of legal authority as exclusionary reasons, does not succeed—either it begs the question of whether law actually provides exclusionary reasons or, if (as Wendel suggests) the reasons are not wholly exclusionary, Raz’s two levels of reasoning collapse into one, and acting on moral grounds is not in fact excluded by legal authority. I then turn to Wendel\u27s ideas about moral remainders —the moral costs that acting on his view of legal ethics may inflict on others. Wendel suggests that some form of atonement can cancel the moral remainder, but I am skeptical that his proposal—atoning through law reform activities—can do the job

    On the Robustness of Explanations of Deep Neural Network Models: A Survey

    Full text link
    Explainability has been widely stated as a cornerstone of the responsible and trustworthy use of machine learning models. With the ubiquitous use of Deep Neural Network (DNN) models expanding to risk-sensitive and safety-critical domains, many methods have been proposed to explain the decisions of these models. Recent years have also seen concerted efforts that have shown how such explanations can be distorted (attacked) by minor input perturbations. While there have been many surveys that review explainability methods themselves, there has been no effort hitherto to assimilate the different methods and metrics proposed to study the robustness of explanations of DNN models. In this work, we present a comprehensive survey of methods that study, understand, attack, and defend explanations of DNN models. We also present a detailed review of different metrics used to evaluate explanation methods, as well as describe attributional attack and defense methods. We conclude with lessons and take-aways for the community towards ensuring robust explanations of DNN model predictions.Comment: Under Review ACM Computing Surveys "Special Issue on Trustworthy AI

    Privacy Meets Explainability: A Comprehensive Impact Benchmark

    Full text link
    Since the mid-10s, the era of Deep Learning (DL) has continued to this day, bringing forth new superlatives and innovations each year. Nevertheless, the speed with which these innovations translate into real applications lags behind this fast pace. Safety-critical applications, in particular, underlie strict regulatory and ethical requirements which need to be taken care of and are still active areas of debate. eXplainable AI (XAI) and privacy-preserving machine learning (PPML) are both crucial research fields, aiming at mitigating some of the drawbacks of prevailing data-hungry black-box models in DL. Despite brisk research activity in the respective fields, no attention has yet been paid to their interaction. This work is the first to investigate the impact of private learning techniques on generated explanations for DL-based models. In an extensive experimental analysis covering various image and time series datasets from multiple domains, as well as varying privacy techniques, XAI methods, and model architectures, the effects of private training on generated explanations are studied. The findings suggest non-negligible changes in explanations through the introduction of privacy. Apart from reporting individual effects of PPML on XAI, the paper gives clear recommendations for the choice of techniques in real applications. By unveiling the interdependencies of these pivotal technologies, this work is a first step towards overcoming the remaining hurdles for practically applicable AI in safety-critical domains.Comment: Under Submissio

    Using Model Explanations to Guide Deep Learning Models Towards Consistent Explanations for EHR Data

    Get PDF
    It has been shown that identical Deep Learning (DL) architectures will produce distinct explanations when trained with different hyperparameters that are orthogonal to the task (e.g. random seed, training set order). In domains such as healthcare and finance, where transparency and explainability is paramount, this can be a significant barrier to DL adoption. In this study we present a further analysis of explanation (in)consistency on 6 tabular datasets/tasks, with a focus on Electronic Health Records data. We propose a novel deep learning ensemble architecture that trains its sub-models to produce consistent explanations, improving explanation consistency by as much as 315% (e.g. from 0.02433 to 0.1011 on MIMIC-IV), and on average by 124% (e.g. from 0.12282 to 0.4450 on the BCW dataset). We evaluate the effectiveness of our proposed technique and discuss the implications our results have for both industrial applications of DL and explainability as well as future methodological work

    EMaP: Explainable AI with Manifold-based Perturbations

    Full text link
    In the last few years, many explanation methods based on the perturbations of input data have been introduced to improve our understanding of decisions made by black-box models. The goal of this work is to introduce a novel perturbation scheme so that more faithful and robust explanations can be obtained. Our study focuses on the impact of perturbing directions on the data topology. We show that perturbing along the orthogonal directions of the input manifold better preserves the data topology, both in the worst-case analysis of the discrete Gromov-Hausdorff distance and in the average-case analysis via persistent homology. From those results, we introduce EMaP algorithm, realizing the orthogonal perturbation scheme. Our experiments show that EMaP not only improves the explainers' performance but also helps them overcome a recently-developed attack against perturbation-based methods.Comment: 29 page

    Investigating sexual coercion in romantic relationships : a test of the cuckoldry risk hypothesis

    Get PDF
    Sexual coercion in romantic relationships is a facet of criminal behaviour requiring psychological investigation. The cuckoldry risk hypothesis, that sexual coercion is a tactic used by some males to reduce the risk of cuckoldry by engaging in sperm competition, was developed to account for such behaviour. From this hypothesis, four predictions were generated and empirically tested: (1) males should be more willing to use sexually coercive tactics when the risk of cuckoldry is high; (2) greater instances of cuckoldry risk in the past should be related to greater instances of sexual aggression; (3) cuckoldry risk and sexual jealousy should positively correlate in men; and (4) among males, rape attitudes and arousal are highest when the risk of cuckoldry is high. Theoretical considerations also suggested the following exploratory questions: (1) are factors currently known to be related to general sexual coercion also related to measures of coercion in romantic relationships; and (2) can the cuckoldry risk measures still predict coercion after controlling for psychopathy? In order to test these predictions, a sample of 82 male and 82 female undergraduate students who were sexually active in a heterosexual relationship completed a survey that collected information on demographics, relationship characteristics, arousal, antisociality, and attitudes. Results found: (1) a significant interaction between cuckoldry risk variables in predicting coercion among male participants and not among females; (2) no relationship between past instances of cuckoldry risk and instances of sexual aggression; (3) those who spend proportionally less time away from their partner were more likely to score higher on sexual jealousy; (4) significant interactions in the anticipated direction were found when predicting scores on the Rape Empathy Scale and Rape Myth Acceptance Scale, a trend in the anticipated direction was found when predicting Adversarial Sexual Beliefs, and nonsignificant results were found when predicting Attraction to Sexual Aggression. Results addressing the exploratory questions found that: (1) only psychopathy significantly predicted partner sexual coercion; and (2) cuckoldry risk variables predicted sexual coercion after controlling for psychopathy. Discussion of these results cover: the importance of finding a sex difference; understanding the interaction between variables; how cuckoldry risk impacts rape-supportive thoughts, attitudes, and arousal; the role of sexual jealousy; the function of a cuckoldry risk psychological mechanism; and lastly, the implications on dynamic risk prediction
    • …
    corecore