Funding Information: Supported by EPSRC DTP Grant Number EP/N509814/1Peer reviewedPublisher PD

Coghill, George M.

Forrest, James

Pang, Wei

Sripada, Somayajulu

English

From the user perspective (data subjects and data controllers), useful explanations of ML decisions are selective, contrastive and social. In this paper, we describe an algorithm for generating selective and contrastive explanations and experimentally study its usefulness to users

Coghill, George

Heriot Watt Pure

Are Contrastive Explanations Useful?

Funding Information: Supported by EPSRC DTP Grant Number EP/N509814/1Peer reviewe

Aberdeen University Research

Are Contrastive Explanations Useful? ?James Forrest1, Somayajulu Sripada1, Wei Pang2, and George M. Coghill11 University of Aberdeen {j.forrest , yaji.sripada , g.coghill} @abdn.ac.uk2 Heriot-Watt University W.Pang@hw.ac.ukAbstract. From the user perspective (data subjects and data controllers),useful explanations of ML decisions are selective, contrastive and social.In this paper, we describe an algorithm for generating selective and con-trastive explanations and experimentally study its usefulness to users.Keywords: Interpretable ML · Contrastive Explanations · XAI.1 IntroductionMachine Learning (ML) models are making ever increasing numbers of decisionsthat impact people’s lives. This makes it important to have explanations thatenable those who develop and deploy ML models and those who are subject totheir decisions to examine these models to ensure that they are effective and fair.Miller 2017 proposes three desiderata for explanations - explanations should beselective, contrastive and social. Explanations should be contrastive, explaininghow other events could have happened rather than explaining the event that didhappen. Explanations should be selective, only presenting information relevantto the recipient. Explanations should be social because they are a communicationbetween explainer and recipient and need to respect the recipient’s needs.Because operationally deployed ML models could come from an extensivetaxonomy of ML models (from decision trees to deep neural networks) an equallylarge taxonomy of interpretation methods have been developed (Guidotti et al.,2018). Not all these interpretation methods fulfil Miller’s desiderata. Wachteret al. 2017 propose that Contrastive Explanations best fulfil these criteria. Ina Contrastive Explanation, the recipient is shown counterfactuals, where thedecision model would make different decisions, as the explanation. This work isa test of some claims made for Contrastive Explanations.In order to have data professionals make better use of explanations, Kaur etal 2020 suggest that ML explanations have improved use of HCI to aid under-standing in users. Especially to promote better alignment of the users’ mentalmodels and the conceptual models of the Interpretable Machine Learning (IML)tool and help the users move from reflexive to deliberative thinking.? Supported by EPSRC DTP Grant Number EP/N509814/1Copyright © 2021 for this paper by its authors. Use permitted under CreativeCommons License Attribution 4.0 International (CC BY 4.0).2 James Forrest et al.In this paper, we present a Contrastive Explanation technique built on opt-aiNet, an optimisation algorithm based on an Immune Inspired Algorithm (Tim-mis and Edmonds, 2010) that fulfils Miller’s desiderata. We evaluate our Con-trastive Explanation technique in comparison to the contrastive tool DiCE (Mothilalet al., 2020), as well as Feature Saliency explanations.2 Explanation GenerationA Contrastive Explanation is a collection of counterfactuals; each counterfactualshows the changes to the original event (fact) that would produce the wantedevent (foil). These counterfactuals are most effective when selective; the moreselective the explanation, the fewer causes the explanation contains, the simplerand more comprehensible the explanation. The counterfactuals should producefoil events that are as near to fact event as possible to be more actionable to therecipient2.1 opt-aiNetOur counterfactual generation algorithm one of a family of algorithms calledArtificial Immune Systems (AIS), which use the Immune System as a metaphorto solve problems. The counterfactuals are generated by the AIS algorithm opt-aiNet, an existing algorithm that we have applied to the domain of explanationgeneration.The features of an opt-aiNet system (illustrated in fig 1) are:– The counterfactuals in this AIS take the cell in the Immune System as theirmetaphor.– The counterfactuals (cells) clone themselves with mutations, a mutationchanges the explanation randomly.– The distance between a cell and the target is the affinity; cells near the targethave high affinity, and cells far from the target have low affinity. Mutationvaries inversely with the affinity; high-affinity cells mutate little while low-affinity cells mutate much more.– Counterfactuals (cells) interact with each other with high-affinity counter-factuals suppressing similar counterfactuals with lower affinity. This removalof similar counterfactuals helps enforce the diversity of counterfactuals.Pseudocode for opt-aiNet (Timmis and Edmonds, 2010) adapted for expla-nation counterfactuals1: Produce initial population of counterfactuals (cells)2: repeat3: repeat4: Generate N clone counterfactuals for each counterfactual. All must beclassified as foil class5: Mutate each counterfactual in inverse proportion to the affinity of theparent counterfactualAre Contrastive Explanations Useful? 3Fig. 1. Illustration of opt-aiNet for Counterfactual Generation6: Determine the affinity of all counterfactuals7: Keep counterfactual with the highest affinity suppress all others8: until average affinity lower than previous iteration9: Determine highest affinity counterfactuals, suppress similar lower affinitycounterfactuals10: Introduce a proportion of new randomly generated counterfactuals11: until stopping condition metThe algorithm is based on the the Ruby programming code provided byBrownlee 2012, which was then reimplemented by us in Python.2.2 DiCEDiCE is an IML tool that produces counterfactual explanations produced in(Mothilal et al., 2020). Optimising on the proximity of the explanation to thefact, this algorithm also optimises to maximise the diversity of the set of expla-nations it creates.3 ExperimentsOur experiments evaluated the explanations of a model’s decision. The evalua-tion methodology followed that suggested in Hoffman et al. 2018 for ExplainableAI (XAI) and Interpretable Machine Learning (IML) evaluations. We used a Testof Performance, in which the participant’s comprehension of the explanation ofthe model’s decision was determined by a proxy task of reversing an adverse4 James Forrest et al.decision of the model. We used a Test of Satisfaction where the participantsanswered questions on their satisfaction with the explanation. We did not usea Test of Comprehension, which evaluates the participant’s mental models, be-cause these tests are very hard to execute well and are hard to use one test acrossdifferent types of explanation, whereas our Test of Performance could be usedfor all the explanation types.The data used in this experiment comes from the lending dataset maintainedby Kaggle at https://www.kaggle.com/wordsforthewise/lending-club, which holdsdata about credit decisions. The decision model used TensorFlow to create aDeep Neural Network.The task simulated a situation where a data subject received a negativeprediction and needed to know the changes to their record that would give apositive prediction. A negative prediction was used in this experiment becausedata subjects are presumed to care about negative credit predictions more thanpositive credit predictions. And gives a natural task of amending the record toget a positive result.Participants were recruited using Amazon Mechanical Turk. The tests wereconducted between explanation types, with each experiment using one explana-tion only. Every participant was shown the same three records with only theexplanations changing between participants.The Test of Performance was to change the values of the record so that theresult changed from a refusal of credit where the model scored the record < 0.5to an acceptance where the model scores ≥ 0.5. The participants were askedto change the record by the smallest amount which would cause the model toclassify it positively. Those participants with the best understanding of the modelshould produce a score ≥ 0.5 but close to 0.5.The Test of Satisfaction was six questions on the user’s satisfaction with theexplanation rated on a five-point Likert scale.The questions were:1. I understand this explanation of how the decision was made.2. This explanation of how the decision was made is satisfying.3. The explanation of how the decision was made has sufficient detail.4. This explanation of how the decision was made contains irrelevant details.5. The record is useful to the goal of having the loan application accepted6. This explanation lets me judge the trustworthiness of the explanation.Further evaluation was done using two Feature Saliency explanations and acontrol of no explanation. Feature Saliency explanations for tabular data producea set of pairs of input Features with their Importance. The tools used to generatethese Feature Saliency explanations were LIME (Ribeiro et al., 2016) and SHAP(Lundberg and Lee, 2017), two very popular tools in IML explanation. Therewere two presentations used of the Feature Saliency explanation type: a bar-chartand a Natural Language Generation (NLG) text. Both presentations have a tableshowing the feature/importance pairs. With an additional element of either thesame data presented as a bar-chart, or a text description. The participants whoreceived No Explanation as a control only saw the data in the record.Are Contrastive Explanations Useful? 54 Results4.1 Test of PerformanceThe Test of Performance was conducted on whether the participant’s alterationsto the record successfully produced a change to the models decision from declinedto accepted (Correct) or not (Incorrect), and evaluated using the Chi-Squaredtest. The results are shown in Table 1. Contrastive Explanations show no sig-nificant difference in whether opt-aiNet or DiCE generate the counterfactuals orwhether one or three counterfactuals were used in the explanation. There is nosignificant difference in performance when the best performing Contrastive ex-planantion (opt-aiNet with three counterfactuals) is compared to other types ofexplanations. Comparing the explanation to ‘No Explanation’ where the partici-pants are only shown the values in the record as a control explanation shows thatthis has the same performance as any given explanation. There are no significantdifferences in performance because this is preliminary work with small numbersof participants. Future work might involve doing these tests of explanations withgreater numbers of participants to achieve significant results.Explanations Total Correct Incorrect % Correct significance from χ2 testwith opt-aiNet with 3cfopt-aiNet with 30 19 11 63% 1one counterfactualDiCE with 27 16 11 59% 0.59one counterfactualopt-aiNet with 30 20 10 67% NAthree counterfactualsDiCE with 30 19 11 63% 1three counterfactualsFeature Saliency 63 41 22 65% 0.934with bar-chartNLG with 60 38 22 63% 0.938tableNo Explanation 60 40 20 67% 1Table 1. Comparison on Test of Performance of different explanations4.2 Test of SatisfactionThe results for the Test of Satisfaction are shown in Table 2, which shows themean values for the six five-point Likert scale questions. The participants hadhigher satisfaction ratings for Contrastive Explanations with three counterfac-tuals rather than one counterfactual. The satisfaction ratings are similar forboth counterfactual generation tools. The opt-aiNet with three counterfactualshas similar satisfaction ratings to the Feature Saliency explanations with either6 James Forrest et al.Explanation Type Q 1 Q 2 Q 3 Q 4 Q 5 Q 6opt-aiNet with 3.93 3.59 * 3.67 3.33 3.77 3.47one counterfactualDiCE with 3.66 3.36 * 3.55 3.48 3.82 3.33one counterfactualopt-aiNet with 3.93 4.27 3.80 3.07 4.13 3.87three counterfactualsDiCE with 4.09 4.34 4.00 3.17 4.17 4.09three counterfactualsFeature Saliency 3.95 4.00 4.06 3.45 3.95 4.12with bar-chartNLG with 4.10 4.08 4.07 3.45 3.95 4.12tableNo Explanation 3.63 3.47 * 3.57 3.22 3.90 3.85Table 2. Mean values of the five point likert questions in the Test of Satisfaction* Significant differences between explanations and opt-AINet with threecounterfactualsbar-chart or NLG. Further, the control ‘No Explanation’ has lower satisfactionratings than these three explanations.The significance of the results was evaluated using Mann-Whitney, comparedto the opt-aiNet with three counterfactuals. The significant differences were forthe second question, ‘This explanation of how the decision was made is satisfy-ing’, where No Explanation and Contrastive Explanations with one counterfac-tual had lower ratings.5 DiscussionThe Test of Performance produced two unexpected results. Firstly, that Con-trastive Explanation with one counterfactual performed unexpectedly poorlycompared to other explanations. Because if the record is changed according tothe values in the counterfactual, then the classification will change. Secondly, theparticipants who received the ‘No Explanation’ of just the values in the recordperformed as well at the task as those who received an explanation, even thoughtheir satisfaction ratings were lower.Examining why this might be, figure 2 shows box-plots of the decision mod-els scores of the participant’s records. The experiment instructions asked par-ticipants to make the smallest changes to the record that would give a positiveclassification. For the Contrastive Explanations the box-plots show the decisionmodel scores are close to the decision boundary at 0.5, but for the other explana-tions the average score is further from the decision boundary and have a largerrange of scores. This shows the Contrastive Explanations produce more optimalresults, whereas, Feature Saliency explanations produce explanations that arestill effective at changing the classification, but do so in a less optimal way (thescore is further from the decision boundary).Are Contrastive Explanations Useful? 7The ‘No Explanation’ is effective at changing the classification but poor atproducing an optimal result, as the average score is the furthest from the de-cision boundary. The ‘No Explanation’ is showing the participants own mentalmodels of the lending domain without being altered by IML explanations. Mostparticipants who saw the ‘No Explanation’ have a good Mental Model about howto change a record to get credit approval e.g. have a high income and good creditgrade. But both participants who saw a Feature Saliency (bar-chart and NLG)explanation and ‘No explanation’ did not know how much to change the recordby to change the record by to get a different classification as this information isnot given in the explanations.Fig. 2. Box-plots of decision model’s scores of participants altered records, for expla-nations6 ConclusionThis preliminary work shows that the opt-aiNet algorithm produces counter-factuals as good as the established algorithm DiCE. Contrastive Explanationsof the model are not necessarily more effective than Feature Saliency explana-tions or just showing the participants the data in the record used by the MLmodel. Contrastive Explanations do enable the participants to make more opti-mal changes to the record, that are nearer to the decision boundary. Participantshave their own mental models of how well known domains, this allows them tobe effective at changing the decisions of classifiers without use of IML explana-tions. Future work will require more data and experiments to discover what theeffects of explanations of ML decisions are on users.BibliographyJ. Brownlee. Clever Algorithms. 2012. ISBN 9781446785065. URL http://www.cleveralgorithms.com.R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, D. Pedreschi, and F. Giannotti.A Survey Of Methods For Explaining Black Box Models. 51(5), 2018. ISSN03600300. https://doi.org/10.1145/3236009. URL http://arxiv.org/abs/1802.01933.R. R. Hoffman, S. T. Mueller, G. Klein, and J. Litman. Metrics for ExplainableAI: Challenges and Prospects. pages 1–50, 2018. URL https://arxiv.org/abs/1812.04608.H. Kaur, H. Nori, S. Jenkins, R. Caruana, H. Wallach, and J. Wort-man Vaughan. Interpreting Interpretability: Understanding Data Scien-tists’ Use of Interpretability Tools for Machine Learning. CHI Conferenceon Human Factors in Computing Systems Proceedings, pages 1–14, 2020.https://doi.org/10.1145/3313831.3376219.S. Lundberg and S.-I. Lee. A Unified Approach to Interpreting Model Predic-tions. 2017. ISSN 10495258. URL https://arxiv.org/abs/1705.07874.T. Miller. Explanation in Artificial Intelligence: Insights from the Social Sciences.2017. https://doi.org/arXiv:1706.07269v1. URL http://arxiv.org/abs/1706.07269.R. K. Mothilal, A. Sharma, and C. Tan. Explaining machine learningclassifiers through diverse counterfactual explanations. In Proceedings ofthe 2020 Conference on Fairness, Accountability, and Transparency, pages607–617, New York, NY, USA, 1 2020. ACM. ISBN 9781450369367.https://doi.org/10.1145/3351095.3372850. URL http://arxiv.org/abs/1905.07697http://dl.acm.org/doi/10.1145/3351095.3372850.M. T. Ribeiro, S. Singh, and C. Guestrin. ”Why Should I Trust You?”: Ex-plaining the Predictions of Any Classifier. 2016. ISSN 9781450321389.https://doi.org/10.1145/1235. URL http://arxiv.org/abs/1602.04938.J. Timmis and C. Edmonds. A Comment on Opt-AiNET: An Im-mune Network Algorithm for Optimisation. (May):308–317, 2010.https://doi.org/10.1007/978-3-540-24854-5 32.S. Wachter, B. Mittelstadt, and L. Floridi. Why a right to explanationof automated decision-making does not exist in the General Data Protec-tion Regulation. International Data Privacy Law, pages 1–47, 2017. URLhttp://arxiv.org/abs/1606.08813.

Are contrastive explanations useful?

https://aura.abdn.ac.uk/bitstream/2164/16964/1/Forrest_etal_are_contrastive_explanations_VOR.pdf

Are contrastive explanations useful?

Abstract

Similar works

Full text

Available Versions

Heriot Watt Pure

Aberdeen University Research

Heriot Watt Pure