36 research outputs found
Tackling Over-Smoothing for General Graph Convolutional Networks
Increasing the depth of GCN, which is expected to permit more expressivity,
is shown to incur performance detriment especially on node classification. The
main cause of this lies in over-smoothing. The over-smoothing issue drives the
output of GCN towards a space that contains limited distinguished information
among nodes, leading to poor expressivity. Several works on refining the
architecture of deep GCN have been proposed, but it is still unknown in theory
whether or not these refinements are able to relieve over-smoothing. In this
paper, we first theoretically analyze how general GCNs act with the increase in
depth, including generic GCN, GCN with bias, ResGCN, and APPNP. We find that
all these models are characterized by a universal process: all nodes converging
to a cuboid. Upon this theorem, we propose DropEdge to alleviate over-smoothing
by randomly removing a certain number of edges at each training epoch.
Theoretically, DropEdge either reduces the convergence speed of over-smoothing
or relieves the information loss caused by dimension collapse. Experimental
evaluations on simulated dataset have visualized the difference in
over-smoothing between different GCNs. Moreover, extensive experiments on
several real benchmarks support that DropEdge consistently improves the
performance on a variety of both shallow and deep GCNs.Comment: Submitted to TPAMI, 15 page
Rumor Detection on Social Media with Bi-Directional Graph Convolutional Networks
Social media has been developing rapidly in public due to its nature of
spreading new information, which leads to rumors being circulated. Meanwhile,
detecting rumors from such massive information in social media is becoming an
arduous challenge. Therefore, some deep learning methods are applied to
discover rumors through the way they spread, such as Recursive Neural Network
(RvNN) and so on. However, these deep learning methods only take into account
the patterns of deep propagation but ignore the structures of wide dispersion
in rumor detection. Actually, propagation and dispersion are two crucial
characteristics of rumors. In this paper, we propose a novel bi-directional
graph model, named Bi-Directional Graph Convolutional Networks (Bi-GCN), to
explore both characteristics by operating on both top-down and bottom-up
propagation of rumors. It leverages a GCN with a top-down directed graph of
rumor spreading to learn the patterns of rumor propagation, and a GCN with an
opposite directed graph of rumor diffusion to capture the structures of rumor
dispersion. Moreover, the information from the source post is involved in each
layer of GCN to enhance the influences from the roots of rumors. Encouraging
empirical results on several benchmarks confirm the superiority of the proposed
method over the state-of-the-art approaches.Comment: 8 pages, 4 figures, AAAI 202
A Restricted Black-box Adversarial Framework Towards Attacking Graph Embedding Models
With the great success of graph embedding model on both academic and industry
area, the robustness of graph embedding against adversarial attack inevitably
becomes a central problem in graph learning domain. Regardless of the fruitful
progress, most of the current works perform the attack in a white-box fashion:
they need to access the model predictions and labels to construct their
adversarial loss. However, the inaccessibility of model predictions in real
systems makes the white-box attack impractical to real graph learning system.
This paper promotes current frameworks in a more general and flexible sense --
we demand to attack various kinds of graph embedding model with black-box
driven. To this end, we begin by investigating the theoretical connections
between graph signal processing and graph embedding models in a principled way
and formulate the graph embedding model as a general graph signal process with
corresponding graph filter. As such, a generalized adversarial attacker:
GF-Attack is constructed by the graph filter and feature matrix. Instead of
accessing any knowledge of the target classifiers used in graph embedding,
GF-Attack performs the attack only on the graph filter in a black-box attack
fashion. To validate the generalization of GF-Attack, we construct the attacker
on four popular graph embedding models. Extensive experimental results validate
the effectiveness of our attacker on several benchmark datasets. Particularly
by using our attack, even small graph perturbations like one-edge flip is able
to consistently make a strong attack in performance to different graph
embedding models.Comment: Accepted by the AAAI 202