Generative Explanations for Graph Neural Network: Methods and Evaluations

Graph Neural Networks (GNNs) achieve state-of-the-art performance in various graph-related tasks. However, the black-box nature often limits their interpretability and trustworthiness. Numerous explainability methods have been proposed to uncover the decision-making logic of GNNs, by generating underlying explanatory substructures. In this paper, we conduct a comprehensive review of the existing explanation methods for GNNs from the perspective of graph generation. Specifically, we propose a unified optimization objective for generative explanation methods, comprising two sub-objectives: Attribution and Information constraints. We further demonstrate their specific manifestations in various generative model architectures and different explanation scenarios. With the unified objective of the explanation problem, we reveal the shared characteristics and distinctions among current methods, laying the foundation for future methodological advancements. Empirical results demonstrate the advantages and limitations of different explainability approaches in terms of explanation performance, efficiency, and generalizability

Bee Colony Optimization Applied to the Bin Packing Problem

We treat the two-dimensional bin packing problem which involves packing a given set of rectangles into a minimum number of larger identical rectangles called bins. This combinatorial problem is NP-hard. We propose a pretreatment for the oriented version of the problem that allows the valorization of the lost areas in the bins and the reduction of the size problem. A heuristic method based on the strategy first-fit adapted to this problem is presented. We present an approach of resolution by bee colony optimization. Computational results express a comparison of the number of bins used with and without pretreatment

GInX-Eval: Towards In-Distribution Evaluation of Graph Neural Network Explanations

Diverse explainability methods of graph neural networks (GNN) have recently been developed to highlight the edges and nodes in the graph that contribute the most to the model predictions. However, it is not clear yet how to evaluate the correctness of those explanations, whether it is from a human or a model perspective. One unaddressed bottleneck in the current evaluation procedure is the problem of out-of-distribution explanations, whose distribution differs from those of the training data. This important issue affects existing evaluation metrics such as the popular faithfulness or fidelity score. In this paper, we show the limitations of faithfulness metrics. We propose GInX-Eval (Graph In-distribution eXplanation Evaluation), an evaluation procedure of graph explanations that overcomes the pitfalls of faithfulness and offers new insights on explainability methods. Using a retraining strategy, the GInX score measures how informative removed edges are for the model and the EdgeRank score evaluates if explanatory edges are correctly ordered by their importance. GInX-Eval verifies if ground-truth explanations are instructive to the GNN model. In addition, it shows that many popular methods, including gradient-based methods, produce explanations that are not better than a random designation of edges as important subgraphs, challenging the findings of current works in the area. Results with GInX-Eval are consistent across multiple datasets and align with human evaluation

Explaining compound activity predictions with a substructure-aware loss for graph neural networks

Explainable machine learning is increasingly used in drug discovery to help rationalize compound property predictions. Feature attribution techniques are popular choices to identify which molecular substructures are responsible for a predicted property change. However, established molecular feature attribution methods have so far displayed low performance for popular deep learning algorithms such as graph neural networks (GNNs), especially when compared with simpler modeling alternatives such as random forests coupled with atom masking. To mitigate this problem, a modification of the regression objective for GNNs is proposed to specifically account for common core structures between pairs of molecules. The presented approach shows higher accuracy on a recently-proposed explainability benchmark. This methodology has the potential to assist with model explainability in drug discovery pipelines, particularly in lead optimization efforts where specific chemical series are investigated.ISSN:1758-294

Explaining compound activity predictions with a substructure-aware loss for graph neural networks

Explainable machine learning is increasingly used in drug discovery to help rationalize compound property predictions. Feature attribution techniques are popular choices to identify which molecular substructures are responsible for a predicted property change. However, established molecular feature attribution methods have so far displayed low performance for popular deep learning algorithms such as graph neural networks (GNNs), especially when compared with simpler modeling alternatives such as random forests coupled with atom masking. To mitigate this problem, in this work a modification of the regression objective for GNNs is proposed to specifically account for common core structures between pairs of molecules. The presented approach showed higher accuracy on a recently-proposed explainability benchmark. This methodology has the potential to assist with model explainability in drug discovery pipelines, particularly in lead optimization efforts where specific chemical series are investigated

Graphframex: Towards systematic evaluation of explainability methods for graph neural networks

As one of the most popular machine learning models today, graph neural networks (GNNs) have attracted intense interest recently, and so does their explainability. Users are increasingly interested in a better understanding of GNN models and their outcomes. Unfortunately, today’s evaluation frameworks for GNN explainability often rely on few inadequate synthetic datasets, leading to conclusions of limited scope due to a lack of complexity in the problem instances. As GNN models are deployed to more mission-critical applications, we are in dire need for a common evaluation protocol of explainability methods of GNNs. In this paper, we propose, to our best knowledge, the first systematic evaluation framework for GNN explainability, considering explainability on three different “user needs”. We propose a unique metric that combines the fidelity measures and classifies explanations based on their quality of being sufficient or necessary. We scope ourselves to node classification tasks and compare the most representative techniques in the field of input-level explainability for GNNs. For the inadequate but widely used synthetic benchmarks, surprisingly shallow techniques such as personalized PageRank have the best performance for a minimum computation time. But when the graph structure is more complex and nodes have meaningful features, gradientbased methods are the best according to our evaluation criteria. However, none dominates the others on all evaluation dimensions and there is always a trade-off. We further apply our evaluation protocol in a case study for frauds explanation on eBay transaction graphs to reflect the production environment.ISSN:2640-349

ReforesTree: A Dataset for Estimating Tropical Forest Carbon Stock with Deep Learning and Aerial Imagery

Forest biomass is a key influence for future climate, and the world urgently needs highly scalable financing schemes, such as carbon offsetting certifications, to protect and restore forests. Current manual forest carbon stock inventory methods of measuring single trees by hand are time, labour, and cost-intensive and have been shown to be subjective. They can lead to substantial overestimation of the carbon stock and ultimately distrust in forest financing. The potential for impact and scale of leveraging advancements in machine learning and remote sensing technologies is promising but needs to be of high quality in order to replace the current forest stock protocols for certifications. In this paper, we present ReforesTree, a benchmark dataset of forest carbon stock in six agro-forestry carbon offsetting sites in Ecuador. Furthermore, we show that a deep learning-based end-to-end model using individual tree detection from low cost RGB-only drone imagery is accurately estimating forest carbon stock within official carbon offsetting certification standards. Additionally, our baseline CNN model outperforms state-of-the-art satellite-based forest biomass and carbon stock estimates for this type of small-scale, tropical agro-forestry sites. We present this dataset to encourage machine learning research in this area to increase accountability and transparency of monitoring, verification and reporting (MVR) in carbon offsetting projects, as well as scaling global reforestation financing through accurate remote sensing.Comment: Accepted paper for the AI for Social Impact Track at the AAAI 202

Oxidation Stability of Diesel/Biodiesel Fuels Measured by a PetroOxy Device and Characterization of Oxidation Products

In the present work, the oxidation stability of diesel, rapeseed (RME), and soybean (SME) fatty acid methyl esters (FAME) and a blend of diesel with 10% (v/v) RME (B10–RME) was studied. Fuel samples were aged in the PetroOxy test device from 383 to 423 K at 7 bar. Experiments were conducted in oxygen excess, and the global kinetic constants were determined. The global kinetic constants for diesel, B10–RME, and RME at 383 K were 7.92 × 10^–6, 2.78 × 10^–5, and 8.87 × 10^–5 s^–1, respectively. The oxidation products formed at different stages of the oxidation were monitored by Fourier transform infrared spectroscopy (FTIR), thermogravimetric analysis–differential thermal analysis (TGA–DTA), and gas chromatography/mass spectrometry (GC/MS). The impact of the FAME nature and level of blending on the kinetic rate constant and the oxidation products was investigated. Results show that RME oxidation forms C₁₉ epoxy as the main oxidation product, in addition to a methyl ester FAME derivative and short-chain oxidation products, such as alkane, alkene, aldehydes, ketones, alcohols, and acids with a carbon number up to C₁₁. The overall amount of oxidation products increases with a higher degradation time. The DTA profile suggests that higher molecular weight products are formed at an advanced level of oxidation. For all highly oxidized fuels, a similar DTA peak was obtained at a temperature of around 573 K, which may suggest the formation of products having similar molecular weights for both diesel and FAME

Unpacking democratic resentment: a qualitative research approach

Discussing the importance of resentment towards political institutions of representative democracy, yet the lack of a deep understanding of the concept and phenomenon, this paper first engages in a conceptual discussion of political or democratic resentment. This conceptual exploration proceeds in three steps. Section 1.1 sums up the philosophical genealogy of political or democratic resentment, helping us to distinguish its moral and sociological acceptation Section 1.2 locates political or democratic resentment alongside close emotions in relation with politics. Section 1.3 delineates resentment by inquiring the similarities and differences with neighboring concepts. This conceptual discussion serves as a roadmap for setting out a strategy to empirically study political or democratic resentment.Section 2.1 discusses in what contexts one can expect to be able to observe political or democratic resentment. In other words, this section proposes parameters for a case selection that allows for gaining the needed deeper understanding of political or democratic resentment. Section 2.2 proposes focus groups as a suitable research method to observe and gain deeper understanding of the political emotions and narratives that are key to political and democratic resentment. subject to further revisions, the paper ultimately aims to make four contributions to the literature. First, defining political or democratic resentment and highlighting its boundaries with related concepts makes it possible to operationalize and empirically study the phenomenon. Second, theorize what experiences and situations engender democratic resentment. These two elements lay the groundwork for future research that aims to assess how citizens’ political behavior is shaped by democratic resentment, and how it relates to other political and societal attitudes and behavior. Third, the study of democratic resentment sheds light on the democratic nature of regimes from the citizen’s perspective, but is also insightful for increasing the resilience of democratic representative institutions. Fourth, by using the concept political or democratic resentment as an illustration, we contribute to the development of research designs and methods suited to capture complex phenomena in political sociology more generally