Ayoobi, H

Chew, K

Ghitu, M

Grewal, D

Mihailescu, I

Potyka, N

Sharma, S

Toni, F

Weng, A

Spiral - Imperial College Digital Repository

Submitted to:ICLP 2023 SystemDemonstrations Track© I. Mihailescu, A. Weng, S. Sharma, M. Ghitu, D. Grewal, K. Chew, H. Ayoobi, N. Potyka & F. ToniThis work is licensed under theCreative Commons Attribution License.PySpArX - A Python library for generating SparseArgumentative eXplanations for neural networksIoana Mihailescu*†icm20@ic.ac.ukAlan Weng*†aw920@ic.ac.ukSiddharth Sharma*†ss2120@ic.ac.ukMihnea Ghitu*†mg2720@ic.ac.ukDilshaan Grewal*†dsg20@ic.ac.ukKhemi Chew*†kjc20@ic.ac.ukHamed Ayoobi‡†h.ayoobi@ic.ac.ukNico Potyka‡†n.potyka@ic.ac.ukFrancesca Toni‡†f.toni@ic.ac.uk1 IntroductionThe increasing use of data-centric techniques in autonomous intelligent systems in areas such as health-care, automotive, and finance raises concerns about their predictions’ reliability, fairness, and trans-parency. In the case of systems empowered by black-box machine learning models, the user has noaccess to the inner mechanics of the systems, thus making it essential to obtain a human-readable inter-face providing explanations of the models’ operations.Explainable Artificial Intelligence (XAI) techniques aim to provide user-friendly explanations for mod-els. SpArX (Sparse Argumentative Explanations for Neural Networks [2]) is an XAI method that faith-fully explains the mechanics of neural networks in the form of Multi-Layer Perceptrons (MLPs) byexploiting the correspondence between MLPs and Quantitative Argumentation Frameworks (QAF) [11].Intuitively, QAFs are a form of argumentation framework amounting to weighted graphs that hold argu-ments as their nodes and relations (of attack or support) between arguments as their edges. To provide anexplainable interface of MLPs using QAFs, the MLPs are first sparsified, through a per-layer clusteringof neurons based on output-similarity. The sparsified MLPs are then transformed into QAFs and thenglobal (input-independent) and local (within the vicinity of a target input) explanations are generated,while faithfully preserving the mechanics of the original MLPs.Building upon SpArX, we have provided users with tools that help them prove and understand howMLPs, trained on specific datasets, work. The tools allow users to customize explanations to their needs.2 System OverviewThe architecture comprises of three primary components, namely a Python library, a GUI for generatingmodels, and a web-based application for visualization. We have provided technical users with a Python*Equal contributors.†Imperial College London, London, United Kingdom‡Supervisors.2 PySpArX - A Python library for generating Sparse Argumentative eXplanations for neural networks.Pip package [7] in which a modular implementation of SpArX can be found, as well as a GUI [5] in whichthey can input their own dataset, and generate a customizable MLP. This is forwarded to our frameworkpipeline, which performs the aforementioned transformations (sparsification and mapping onto a QAF).The final QAF is then stored in a database and made available online to other users via a shareable link[10] where it can be visualised.2.1 sparx-libThe Python Pip package, sparx-lib [7], is a custom library developed to handle the heavy computationsrequired for generating SpArX explanations. It is designed with a modular architecture that allows forthe flexible integration of different components of the SpArX algorithm. It enables users to generateexplanations for a variety of models, which can be either user-specified, with a dataset, or imported.Tensorflow and Keras are utilized to develop and train custom models. Users can generate two types ofvisualisations: pure JSON, which can be viewed as text or parsed, or a GUI using NetworkX and Bokeh.We used a documentation generation tool called Sphinx to create documentation for the sparx-lib pippackage [6].2.2 Model generation GUIThe model generation GUI [5] is a tool developed using the React framework. It provides a platformfor administrators or technical users to design MLPs for a selected dataset, generate explanations, andstore visualizations in a remote database for access by other users. The model generation process iscurrently performed locally due to the computational demands involved in training the model and calcu-lating explanations. The requirements and instructions for its execution are provided in the SpArX-GUIrepository [5].Figure 1: Web app visualisation [10] for the Iris dataset [3]I. Mihailescu, A. Weng, S. Sharma, M. Ghitu, D. Grewal, K. Chew, H. Ayoobi, N. Potyka & F. Toni 32.3 VisualisationThe customizable visualization component [9] provides features such as:• displaying node and edge information such as attackers/supporters and their weight using hover-cards, and visualising node dependencies using charts;• filtering graph information by progressively rendering layers and restricting the nodes shown;• sharing generated visualisations via a link/QR code.We have utilised React with React Flow to create interactive graphs that can be easily modified (SeeFigure 1). Our server has been deployed on the web to enhance accessibility to the app. Administratorscan simply share a link to the dedicated visualisation page with other non-technical users.3 Demo requirementsTo run the project, follow the instructions in the README file from the SpArX-GUI repository [5]. Themodel generation GUI accepts datasets in CSV format where the first line represents the name of theparameter/column. A test dataset is provided in the SpArX-GUI repository under data/iris.data. Fordemo purposes, an already generated visualisation of a SpArX explanation can be found online [10]. Ademonstration video can be found at [4].References[1] H. Ayoobi, N. Potyka & F. Toni (2022): SpArX legacy code. Available at https://github.com/H-Ayoobi/SpArX.[2] H. Ayoobi, N. Potyka & F. Toni (2023): SpArX: Sparse Argumentative Explanations for Neural Networks,doi:10.48550/ARXIV.2301.09559. Available at https://arxiv.org/abs/2301.09559.[3] Dheeru Dua & Casey Graff (2017): UCI Machine Learning Repository. Available at http://archive.ics.uci.edu/ml.[4] I. Mihailescu (2023): PySpArX - A library for SpArX explanations of Neural Networks - Demo. Available athttps://youtu.be/OJHl4zegFvU.[5] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): SpArX-GUI. Available athttps://github.com/SpArX-Group-10/SpArX-GUI.[6] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): Sparx-lib documentation.Available at https://sparx-group-10.github.io/SpArX/.[7] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): Sparx-lib pip package. Availableat https://pypi.org/project/sparx-lib/.[8] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): SpArX-lib Repository. Availableat https://github.com/SpArX-Group-10/SpArX.[9] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): SpArX-Visualisation. Availableat https://github.com/SpArX-Group-10/SpArX-Visualisation.[10] I. Mihailescu, K. Chew, M. Ghitu, S. Sharma, D. Grewal & A. Weng (2022): SpArX-Visualisation Web appexample. Available at https://sparx-vis.herokuapp.com/63ac3bd37963d9af247d8102.[11] Nico Potyka (2021): Interpreting Neural Networks as Quantitative Argumentation Frameworks. In: AAAIConference on Artificial Intelligence, (AAAI), AAAI Press, pp. 6463–6470.

Electronic Proceedings in Theoretical Computer Science

English

PySpArX - A Python library for generating Sparse Argumentative eXplanations for neural networks

PySpArX - A Python library for generating Sparse Argumentative eXplanations for neural networks

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository