Färber, Michael

Hiemstra D., Mothe J. Perego R. Potthast M. Sebastiani F., Moens M.

Nguyen, Anna

Oberföll, Adrian

English

KITopen

Right for the Right Reasons: Making ImageClassification Intuitively ExplainableAnna Nguyen , Adrian Oberföll, and Michael FärberKarlsruhe Institute of Technology (KIT), Karlsruhe, Germanyanna.nguyen@kit.eduadrian.oberfoell@student.kit.edumichael.faerber@kit.eduAbstract. The effectiveness of Convolutional Neural Networks (CNNs)in classifying image data has been thoroughly demonstrated. In order toexplain the classification to humans, methods for visualizing classificationevidence have been developed in recent years. These explanations revealthat sometimes images are classified correctly, but for the wrong reasons,i.e., based on incidental evidence. Of course, it is desirable that imagesare classified correctly for the right reasons, i.e., based on the actualevidence. To this end, we propose a new explanation quality metric tomeasure object al igned explanation in image classification which we referto as the ObAlEx metric. Using object detection approaches, explanationapproaches, and ObAlEx, we quantify the focus of CNNs on the actualevidence. Moreover, we show that additional training of the CNNs canimprove the focus of CNNs without decreasing their accuracy.1 IntroductionConvolutional Neural Networks (CNNs) have been demonstrated to be very ef-fective in image classification tasks, achieving high accuracy. However, methodsto explain classifications performed by CNNs have shown that sometimes imagedata has been classified for incidental evidences, undermining the trust betweenhumans and machines [7]. Previous attempts to fix this problem have included ahuman-in-the-loop approach [10], a pre-processing step for removing features ofthe input that are deemed irrelevant for the classification task at hand (such asimages’ backgrounds) [5], or the introduction of a new loss function that incor-porates an explanation approach during training [8]. Although the latter workconstrains the explanation of the model in the loss function penalizing the inputgradients, it uses explanations only based on input gradients which is not idealfor all use cases, especially in image classification, where individual pixels aredifficult to interpret. Overall, we believe that there is a lack of a metric whichquantifies if an intuitive explanation can be gained.In this paper, we propose an object al igned explanation quality metric, calledObAlEx. ObAlEx quantifies to which degree the object mask of an image is con-sistent with the obtained evidence of explanation methods and thus, imitates2 A. Nguyen et al.Input ImageQualityScoreObject Detection(e.g. Mask R-CNN)Image Classifier(e.g. VGG16)Explainer(e.g. Grad-CAM)Desired ExplanationExplanationInputInput ImageQualityScoreObject Detection(e.g. Mask R-CNN)Image Classifier(e.g. VGG16)Explainer(e.g. Grad-CAM)Desired ExplanationExplanationObject DetectionImage Classifica6onFeature ExtractionExplana6on Method ExplanationObjectMaskInput ImageQualityScoreObject Detection(e.g. Mask R-CNN)Image Classifier(e.g. VGG16)Explainer(e.g. Grad-CAM)Desired ExplanationExplanation[w1,…,wx]Explanation QualityObAlExFig. 1. The pipeline of our metric.human behavior to classify images according to the objects contained. The pro-posed metric is independent of the used explanation method (e.g., occlusion [13],LIME [7], or Grad-cam [11]) and object detection method and can therefore beapplied together with arbitrary explanation methods and object detection meth-ods. Our approach to identify the focus on the relevant input regions requiresneither human interaction nor pre-processing. Based on extensive experiments,we demonstrate the effectiveness of the proposed metric while training CNNs,ensuring both high accuracy and a focus on the relevant input regions.Our main contributions are as follows:1. We propose an object aligned explanation metric, ObAlEx, to quantify ex-planations of image classification models intuitively. Our metric is applicableto different explanation methods and neither requires human interaction norinterference in the model’s architecture.2. In extensive experiments,1 we show that our metric can be used for mak-ing CNN models for image classification more intuitive while keeping theaccuracy.In the following section, we outline our metric. We then present our extensiveexperiments. Finally, we close with some concluding remarks.2 ObAlEx MetricThe metric ObAlEx is designed as a relative metric which depends on the expla-nation method and the classifier used. Based on the change of the explanationquality during training, it can be evaluated if a certain training strategy leads toan improvement or deterioration of the model’s intuitive explanation. By expla-nation quality, we define the degree of alignment between object to be classifiedand explanation of the classification model.The pipeline to calculate ObAlEx is outlined in Fig. 1. Given an input imageon which an object should be detected, we first apply an object detection method1 We provide the source code online at https://github.com/annugyen/ObAlExMaking Image Classification Intuitively Explainable 3(e.g., Mask R-CNN) to obtain the image regions of the object itself (i.e., objectmask). We define regions of the explanation that lie outside of the object mask asindicative of a classification for the wrong reasons, and conversely, that regions ofthe explanation that lie inside of the object mask as indicative of a classificationfor the right reasons. The mask of objects on images can be obtained with a highaccuracy nowadays (see Sec. 3).Simultaneously, an image classifier (e.g., pretrained VGG16) is applied toobtain labels of recognized objects (e.g., ”dog”). An explanation method (e.g.,Grad-Cam) then outputs the image regions which are most influential given theextracted features from the CNN and the input image.Both the object mask and the explanation output is then used to compute themetric ObAlEx and thus, to improve the explanation quality. Since existing ex-planation methods support different highlighting levels, our score is constructedin such a way that the score is the higher the more of the highlighted explanationaligns with the object mask. In the following, we describe the computation ofthe explanation quality formally.Given a data set D with correctly classified images and an image d ∈ D withpixels pdij , width wd, and height hd, let Ad denote the matrix whose values adijequals the activation of the pixels of the object mask, where i ∈ {1, . . . , hd},j ∈ {1, . . . , wd}, hd, wd ∈ N. We regard Ad as a fuzzy set, i.e. whose values havedegrees of membership depicted as adij . We define adij ∈ R with 0 ≤ adij ≤ 1. Inour experiments, we set adij = 1 if the pixel pdij of the input image belongs tothe object mask and adij = 0, otherwise. Similarly, let Bd be the matrix whosevalues bdij equals the activation of the pixels of the explanation. We additionallynormalize the values bdij between zero and one, i.e. 0 ≤ bdij ≤ 1 where bdij = 1 ifthe pixel pdij of the input image belongs to the highest activation and bdij = 0otherwise. Our metric ObAlEx is, then, defined as follows:ObAlEx(Ad, Bd) =∑i,j adijbdij∑i,j bdij∈ [0, 1] (1)To get the explanation quality of an image classifier, ObAlEx can be appliedon all images in a data set D. We then calculate the average of all values ofthe explanation quality of each picture for an image collection. In doing so, weweight all images equally. The explanation quality of the classifier is defined asAvgObAlEx(D) =1nn∑d=1ObAlEx(Ad, Bd) ∈ [0, 1], (2)where n ∈ N is the number of images in data set D. AvgObAlEx only considersthe scores of images classified correctly by the model, otherwise the metric wouldget skewed. Therefore, images which are classified wrong are excluded.4 A. Nguyen et al.3 Evaluation3.1 Evaluation SettingTo evaluate ObAlEx, we apply pre-trained CNN models. We focus on threestate-of-the-art image classification models: VGG16 [12], ResNet50 [3], and Mo-bileNet [4]. The models are pre-trained on the ILSVRC2012 data set [9] whichis also known as ImageNet. We adapt each model’s upper output dense layers tothe specific data set (i.e., number of categories in the used image classificationdata sets Dogs vs. Cats and Caltech 101, respectively). To show the universalapplicability of ObAlEx, we use different well-known explanation methods suchas occlusion [13], LIME [7], Grad-Cam [11], and Grad-Cam++ [1]. In our exper-iments, the AvgObAlEx settled around a fixed value after 50 images. For thatreason and due to high computing power costs in case of LIME, we calculate theAvgObAlEx for 50 images per epoch in the following experiments. Our experi-ments are executed on a server with 12 GB of GPU RAM. We use TensorFlowand the Keras deep learning library for implementation. We use the followingdata sets in our evaluation:Dogs vs. Cats data set2 contains 3,000 dog and cat images, 1,500 per class.We use Mask R-CNN [2] to create the object masks. The quality of the objectmasks is important for the validity of the proposed metric ObAlEx. Therefore,we manually evaluated the computed object masks for 200 randomly chosenimages regarding the overlap of the whole object. The accuracy was 91%. Thus,we argue that the pre-trained Mask R-CNN performs well for our purpose.Given the data set size, we used 70% of the images for training and 30% fortesting. We first adjust the output layer of all CNN models to the two categories(dog and cat) and train them for 10 epochs on the Dogs vs. Cats data set (whereall layers except output layer are frozen). After that, we freeze different combina-tions of layers for further training. In the original papers of the above mentionedmodels, the convolutional layers are divided into five blocks. For simplificationand comparability, we use this convention for our strategies. We also summarizethe last dense layers to one block. Thus, we always set whole blocks of layersto either be trainable or non-trainable. We train every strategy for another 10epochs. We investigate the following strategies: (a) train the last dense layerswhich we denote as dense block, (b) train the last two convolutional blocks (i.e.the fourth and fifth), (c) train the first three convolutional blocks, and (d) trainall layers, i.e. all convolutional and dense blocks.Caltech 101 data set [6] has 101 object categories. We create a uniform dis-tributed data set by drawing random sampling from the categories resulting ina total of 6,060 images with 60 images per class. We use a test split of 0.25. Thisdata set is provided with hand-labeled object masks for all images. Thus, we usethose labeled object masks. We perform another experiment inspired by [8,10].2 https://www.kaggle.com/c/dogs-vs-cats, last accessed: 2020-10-28Making Image Classification Intuitively Explainable 5AvgObAlExAvgObAlExAvgObAlExAvgObAlExAvgObAlExAvgObAlExAvgObAlExAvgObAlExFig. 2. VGG16 Results. Transfer learning strategies with VGG16 with explanationmethods Occlusion, LIME and Grad-Cam/Grad-Cam++.To actively force the model to be more intuitive and thus, to provide a more in-terpretable explanation, we followed a näıve approach by using artificial images.We edit the images in a way that they contain the object to classify and maskedout the background with random pixels. This should force the model to focusmore on the object and increase the explanation quality.3.2 Evaluation ResultsDogs vs. Cats. Fig. 2 shows the results for VGG16 with training strategies (a)and (b). We can see that the performance of the model measured with accuracydid not change within 10 epochs (see Fig. 2 (a)/(b) left graph). However, we ob-served a change in AvgObAlEx (see Fig. 2 (a)/(b) right graph). The explanationquality after 10 epochs computed with any explanation method for strategy (b)is significantly higher than the explanation quality for strategy (a). This fits tothe common knowledge that complex structures in the input images are learnedin the later convolutional blocks and are, therefore, more decisive for the classi-fication. Moreover, Fig. 2 (b) shows with increasing number of epochs a decreasein the loss, while the AvgObAlEx increases simultaneously. This indicates theeffectiveness of the model for right predictions based on the right reasons. Theresults of strategy (d) and (b) and the results of strategy (c) and (a) are similarto each other respectively, which emphasizes the common knowledge. Withoutusing the proposed metric ObAlEx this improvement would not be evident sincethe accuracy of all models stays the same during training.6 A. Nguyen et al.0.6509 0.56070.7271 0.6364cat dogbeforetrftertr(i)0.4221 0.41200.8010 0.9575revolver stop signoriginalinain(ii)Fig. 3. Examples from (i) Dogsvs. Cats and (ii) Caltech 101 withquality scores shown above.In Fig. 3 (i) , we provide an example of theexplanation visualized with Grad-Cam withstrategy (b) on VGG16. We can see that theexplanation quality increases after training andthat the visualized explanation has a strongerfocus on the object. With only 10 epochs of ad-ditional training, we were able to improve themodel in a way that it utilizes more importantfeatures such as the face of the animal. With-out ObAlEx, it would be obvious to not trainthe model any further due to the non-changingaccuracy. We observed similar results on the ex-periments with ResNet50 and MobileNet, andalso on the Caltech 101 data set but omit themdue to page limitations.1Caltech 101 Fig. 4 shows the results for 10 epochs of training VGG16 on Caltech101 with the original and masked images as input. As we can observe in the leftFig. 4. Training on Caltech 101.graph, training with the origi-nal images results in a higheraccuracy than training with themasked images. However, theAvgObAlEx (computed withGrad-Cam as explainer, seegraph on the right) of the modeltrained with masked input im-ages is significantly higher thanthe AvgObAlEx of the modeltrained with the original input images. This indicates that more backgroundinformation was used in the classification. Thus, evaluating image classifiers be-yond accuracy can be valuable to real-world cases where specific backgroundinformation is unavailable.Fig. 3 (ii) shows an example image with Grad-Cam on VGG16. Despitehigh accuracy, we can see that the explanation for the image with masked outbackground (image at the bottom) is more intuitive and more focused on theactual object than the original input image.4 ConclusionIn this paper, we focused on evaluating CNN image classifiers with differentexplanation approaches. We introduced a novel explanation quality score metricto support the training process besides accuracy and loss function. We haveshown in our experiments that our metric ObAlEx can be used to indicate caseswhere a model makes its predictions based on wrong reasons. Overall, ObAlExfacilitates more generalized models which can increase the user’s trust in themodel by object aligned explanations.Making Image Classification Intuitively Explainable 7References1. Chattopadhyay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-cam++: Generalized gradient-based visual explanations for deep convolutionalnetworks. In: 2018 IEEE Winter Conference on Applications of Computer Vision,WACV 2018. pp. 839–847. IEEE Computer Society (2018)2. He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. IEEE Transactionson Pattern Analysis and Machine Intelligence 42(2), 386–397 (2020)3. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition.In: 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR2016. pp. 770–778. IEEE Computer Society (2016)4. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., An-dreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks formobile vision applications. CoRR abs/1704.04861 (2017)5. Jia, S., Lansdall-Welfare, T., Cristianini, N.: Right for the right reason: Trainingagnostic networks. In: Advances in Intelligent Data Analysis XVII - 17th Interna-tional Symposium, IDA 2018. Lecture Notes in Computer Science, vol. 11191, pp.164–174. Springer (2018)6. Li, F., Fergus, R., Perona, P.: Learning generative visual models from few train-ing examples: An incremental bayesian approach tested on 101 object categories.Computer Vision and Image Understanding 106(1), 59–70 (2007)7. Ribeiro, M.T., Singh, S., Guestrin, C.: ”why should I trust you?”: Explaining thepredictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD Interna-tional Conference on Knowledge Discovery and Data Mining, 2016. pp. 1135–1144.ACM (2016)8. Ross, A.S., Hughes, M.C., Doshi-Velez, F.: Right for the right reasons: Trainingdifferentiable models by constraining their explanations. In: Proceedings of theTwenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017.pp. 2662–2670. ijcai.org (2017)9. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z.,Karpathy, A., Khosla, A., Bernstein, M.S., Berg, A.C., Li, F.: Imagenet large scalevisual recognition challenge. International Journal of Computer Vision 115(3),211–252 (2015)10. Schramowski, P., Stammer, W., Teso, S., Brugger, A., Shao, X., Luigs, H., Mahlein,A., Kersting, K.: Right for the wrong scientific reasons: Revising deep networks byinteracting with their explanations. CoRR abs/2001.05371 (2020)11. Selvaraju, R.R., Das, A., Vedantam, R., Cogswell, M., Parikh, D., Batra, D.: Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization. CoRR abs/1610.02391 (2016)12. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scaleimage recognition. In: 3rd International Conference on Learning Representations,ICLR 2015, Conference Track Proceedings (2015)13. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks.In: Computer Vision - ECCV 2014 - 13th European Conference. Lecture Notes inComputer Science, vol. 8689, pp. 818–833. Springer (2014)

Right for the Right Reasons: Making Image Classification Intuitively Explainable

https://publikationen.bibliothek.kit.edu/1000134353/149934414

Right for the Right Reasons: Making Image Classification Intuitively Explainable

Abstract

Similar works

Full text

Available Versions

KITopen