363 research outputs found
Revisiting Heterophily in Graph Convolution Networks by Learning Representations Across Topological and Feature Spaces
Graph convolution networks (GCNs) have been enormously successful in learning
representations over several graph-based machine learning tasks. Specific to
learning rich node representations, most of the methods have solely relied on
the homophily assumption and have shown limited performance on the
heterophilous graphs. While several methods have been developed with new
architectures to address heterophily, we argue that by learning graph
representations across two spaces i.e., topology and feature space GCNs can
address heterophily. In this work, we experimentally demonstrate the
performance of the proposed GCN framework over semi-supervised node
classification task on both homophilous and heterophilous graph benchmarks by
learning and combining representations across the topological and the feature
spaces.Comment: Under Review Project Page:
https://sites.google.com/iitgn.ac.in/hetgcn/hom
GPNet: Simplifying Graph Neural Networks via Multi-channel Geometric Polynomials
Graph Neural Networks (GNNs) are a promising deep learning approach for
circumventing many real-world problems on graph-structured data. However, these
models usually have at least one of four fundamental limitations:
over-smoothing, over-fitting, difficult to train, and strong homophily
assumption. For example, Simple Graph Convolution (SGC) is known to suffer from
the first and fourth limitations. To tackle these limitations, we identify a
set of key designs including (D1) dilated convolution, (D2) multi-channel
learning, (D3) self-attention score, and (D4) sign factor to boost learning
from different types (i.e. homophily and heterophily) and scales (i.e. small,
medium, and large) of networks, and combine them into a graph neural network,
GPNet, a simple and efficient one-layer model. We theoretically analyze the
model and show that it can approximate various graph filters by adjusting the
self-attention score and sign factor. Experiments show that GPNet consistently
outperforms baselines in terms of average rank, average accuracy, complexity,
and parameters on semi-supervised and full-supervised tasks, and achieves
competitive performance compared to state-of-the-art model with inductive
learning task.Comment: 15 pages, 15 figure
HeteroNet: Heterophily-aware Representation Learning on Heterogenerous Graphs
Real-world graphs are typically complex, exhibiting heterogeneity in the
global structure, as well as strong heterophily within local neighborhoods.
While a growing body of literature has revealed the limitations of common graph
neural networks (GNNs) in handling homogeneous graphs with heterophily, little
work has been conducted on investigating the heterophily properties in the
context of heterogeneous graphs. To bridge this research gap, we identify the
heterophily in heterogeneous graphs using metapaths and propose two practical
metrics to quantitatively describe the levels of heterophily. Through in-depth
investigations on several real-world heterogeneous graphs exhibiting varying
levels of heterophily, we have observed that heterogeneous graph neural
networks (HGNNs), which inherit many mechanisms from GNNs designed for
homogeneous graphs, fail to generalize to heterogeneous graphs with heterophily
or low level of homophily. To address the challenge, we present HeteroNet,
a heterophily-aware HGNN that incorporates both masked metapath prediction and
masked label prediction tasks to effectively and flexibly handle both
homophilic and heterophilic heterogeneous graphs. We evaluate the performance
of HeteroNet on five real-world heterogeneous graph benchmarks with varying
levels of heterophily. The results demonstrate that HeteroNet outperforms
strong baselines in the semi-supervised node classification task, providing
valuable insights into effectively handling more complex heterogeneous graphs.Comment: Preprin
From Node Interaction to Hop Interaction: New Effective and Scalable Graph Learning Paradigm
Existing Graph Neural Networks (GNNs) follow the message-passing mechanism
that conducts information interaction among nodes iteratively. While
considerable progress has been made, such node interaction paradigms still have
the following limitation. First, the scalability limitation precludes the wide
application of GNNs in large-scale industrial settings since the node
interaction among rapidly expanding neighbors incurs high computation and
memory costs. Second, the over-smoothing problem restricts the discrimination
ability of nodes, i.e., node representations of different classes will converge
to indistinguishable after repeated node interactions. In this work, we propose
a novel hop interaction paradigm to address these limitations simultaneously.
The core idea of hop interaction is to convert the target of message-passing
from nodes into multi-hop features inside each node. Specifically, it first
pre-computed multi-hop features of nodes to reduce computation costs during
training and inference. Then, it conducts a non-linear interaction among
multi-hop features to enhance the discrimination of nodes. We design a simple
yet effective HopGNN framework that can easily utilize existing GNNs to achieve
hop interaction. Furthermore, we propose a multi-task learning strategy with a
self-supervised learning objective to enhance HopGNN. We conduct extensive
experiments on 12 benchmark datasets in a wide range of domains, scales, and
smoothness of graphs. Experimental results show that our methods achieve
superior performance while maintaining high scalability and efficiency
Diffusion-Jump GNNs: Homophiliation via Learnable Metric Filters
High-order Graph Neural Networks (HO-GNNs) have been developed to infer
consistent latent spaces in the heterophilic regime, where the label
distribution is not correlated with the graph structure. However, most of the
existing HO-GNNs are hop-based, i.e., they rely on the powers of the transition
matrix. As a result, these architectures are not fully reactive to the
classification loss and the achieved structural filters have static supports.
In other words, neither the filters' supports nor their coefficients can be
learned with these networks. They are confined, instead, to learn combinations
of filters. To address the above concerns, we propose Diffusion-jump GNNs a
method relying on asymptotic diffusion distances that operates on jumps. A
diffusion-pump generates pairwise distances whose projections determine both
the support and coefficients of each structural filter. These filters are
called jumps because they explore a wide range of scales in order to find bonds
between scattered nodes with the same label. Actually, the full process is
controlled by the classification loss. Both the jumps and the diffusion
distances react to classification errors (i.e. they are learnable).
Homophiliation, i.e., the process of learning piecewise smooth latent spaces in
the heterophilic regime, is formulated as a Dirichlet problem: the known labels
determine the border nodes and the diffusion-pump ensures a minimal deviation
of the semi-supervised grouping from a canonical unsupervised grouping. This
triggers the update of both the diffusion distances and, consequently, the
jumps in order to minimize the classification error. The Dirichlet formulation
has several advantages. It leads to the definition of structural heterophily, a
novel measure beyond edge heterophily. It also allows us to investigate links
with (learnable) diffusion distances, absorbing random walks and stochastic
diffusion
On the Exploitation of Heterophily in Graph-Based Multimodal Remote Sensing Data Analysis
The field of Earth observation is dealing with increasingly large, multimodal data sets. An important processing step consists of providing these data sets with labels. However, standard label propagation algorithms cannot be applied to multimodal remote sensing data for two reasons. First, multimodal data is heterogeneous while classic label propagation algorithms assume a homogeneous network. Second, real-world data can show both homophily ('birds of a feather flock together') and heterophily ('opposites attract') during propagation, while standard algorithms only consider homophily. Both shortcomings are addressed in this work and the result is a graph-based label propagation algorithm for multimodal data that includes homophily and/or heterophily. Furthermore, the method is also able to transfer information between uni- and multimodal data. Experiments on the remote sensing data set of Houston, which contains a LiDAR and a hyperspectral image, show that our approach ties state-of-the-art methods for classification with an OA of 91.4%, while being more flexible and not constrained to a specific data set or a specific combination of modalities.</p
- …