94,200 research outputs found
Graph Neural Networks Use Graphs When They Shouldn't
Predictions over graphs play a crucial role in various domains, including
social networks, molecular biology, medicine, and more. Graph Neural Networks
(GNNs) have emerged as the dominant approach for learning on graph data.
Instances of graph labeling problems consist of the graph-structure (i.e., the
adjacency matrix), along with node-specific feature vectors. In some cases,
this graph-structure is non-informative for the predictive task. For instance,
molecular properties such as molar mass depend solely on the constituent atoms
(node features), and not on the molecular structure. While GNNs have the
ability to ignore the graph-structure in such cases, it is not clear that they
will. In this work, we show that GNNs actually tend to overfit the
graph-structure in the sense that they use it even when a better solution can
be obtained by ignoring it. We examine this phenomenon with respect to
different graph distributions and find that regular graphs are more robust to
this overfitting. We then provide a theoretical explanation for this
phenomenon, via analyzing the implicit bias of gradient-descent-based learning
of GNNs in this setting. Finally, based on our empirical and theoretical
findings, we propose a graph-editing method to mitigate the tendency of GNNs to
overfit graph-structures that should be ignored. We show that this method
indeed improves the accuracy of GNNs across multiple benchmarks
Graph Neural Network for Stress Predictions in Stiffened Panels Under Uniform Loading
Machine learning (ML) and deep learning (DL) techniques have gained
significant attention as reduced order models (ROMs) to computationally
expensive structural analysis methods, such as finite element analysis (FEA).
Graph neural network (GNN) is a particular type of neural network which
processes data that can be represented as graphs. This allows for efficient
representation of complex geometries that can change during conceptual design
of a structure or a product. In this study, we propose a novel graph embedding
technique for efficient representation of 3D stiffened panels by considering
separate plate domains as vertices. This approach is considered using Graph
Sampling and Aggregation (GraphSAGE) to predict stress distributions in
stiffened panels with varying geometries. A comparison between a
finite-element-vertex graph representation is conducted to demonstrate the
effectiveness of the proposed approach. A comprehensive parametric study is
performed to examine the effect of structural geometry on the prediction
performance. Our results demonstrate the immense potential of graph neural
networks with the proposed graph embedding method as robust reduced-order
models for 3D structures.Comment: 20 pages; 7 figure
Spectral Adversarial Training for Robust Graph Neural Network
Recent studies demonstrate that Graph Neural Networks (GNNs) are vulnerable
to slight but adversarially designed perturbations, known as adversarial
examples. To address this issue, robust training methods against adversarial
examples have received considerable attention in the literature.
\emph{Adversarial Training (AT)} is a successful approach to learning a robust
model using adversarially perturbed training samples. Existing AT methods on
GNNs typically construct adversarial perturbations in terms of graph structures
or node features. However, they are less effective and fraught with challenges
on graph data due to the discreteness of graph structure and the relationships
between connected examples. In this work, we seek to address these challenges
and propose Spectral Adversarial Training (SAT), a simple yet effective
adversarial training approach for GNNs. SAT first adopts a low-rank
approximation of the graph structure based on spectral decomposition, and then
constructs adversarial perturbations in the spectral domain rather than
directly manipulating the original graph structure. To investigate its
effectiveness, we employ SAT on three widely used GNNs. Experimental results on
four public graph datasets demonstrate that SAT significantly improves the
robustness of GNNs against adversarial attacks without sacrificing
classification accuracy and training efficiency.Comment: Accepted by TKDE. Code availiable at
https://github.com/EdisonLeeeee/SA
GCondNet: A Novel Method for Improving Neural Networks on Small High-Dimensional Tabular Data
Neural network models often struggle with high-dimensional but small
sample-size tabular datasets. One reason is that current weight initialisation
methods assume independence between weights, which can be problematic when
there are insufficient samples to estimate the model's parameters accurately.
In such small data scenarios, leveraging additional structures can improve the
model's training stability and performance. To address this, we propose
GCondNet, a general approach to enhance neural networks by leveraging implicit
structures present in tabular data. We create a graph between samples for each
data dimension, and utilise Graph Neural Networks (GNNs) for extracting this
implicit structure, and for conditioning the parameters of the first layer of
an underlying predictor MLP network. By creating many small graphs, GCondNet
exploits the data's high-dimensionality, and thus improves the performance of
an underlying predictor network. We demonstrate the effectiveness of our method
on nine real-world datasets, where GCondNet outperforms 14 standard and
state-of-the-art methods. The results show that GCondNet is robust and can be
applied to any small sample-size and high-dimensional tabular learning task.Comment: Early version presented at the 17th Machine Learning in Computational
Biology (MLCB) meeting, 202
Robust Graph Representation Learning via Predictive Coding
Predictive coding is a message-passing framework initially developed to model
information processing in the brain, and now also topic of research in machine
learning due to some interesting properties. One of such properties is the
natural ability of generative models to learn robust representations thanks to
their peculiar credit assignment rule, that allows neural activities to
converge to a solution before updating the synaptic weights. Graph neural
networks are also message-passing models, which have recently shown outstanding
results in diverse types of tasks in machine learning, providing
interdisciplinary state-of-the-art performance on structured data. However,
they are vulnerable to imperceptible adversarial attacks, and unfit for
out-of-distribution generalization. In this work, we address this by building
models that have the same structure of popular graph neural network
architectures, but rely on the message-passing rule of predictive coding.
Through an extensive set of experiments, we show that the proposed models are
(i) comparable to standard ones in terms of performance in both inductive and
transductive tasks, (ii) better calibrated, and (iii) robust against multiple
kinds of adversarial attacks.Comment: 27 Pages, 31 Figure
Dynamic Inference on Graphs using Structured Transition Models
Enabling robots to perform complex dynamic tasks such as picking up an object
in one sweeping motion or pushing off a wall to quickly turn a corner is a
challenging problem. The dynamic interactions implicit in these tasks are
critical towards the successful execution of such tasks. Graph neural networks
(GNNs) provide a principled way of learning the dynamics of interactive systems
but can suffer from scaling issues as the number of interactions increases.
Furthermore, the problem of using learned GNN-based models for optimal control
is insufficiently explored. In this work, we present a method for efficiently
learning the dynamics of interacting systems by simultaneously learning a
dynamic graph structure and a stable and locally linear forward model of the
system. The dynamic graph structure encodes evolving contact modes along a
trajectory by making probabilistic predictions over the edges of the graph.
Additionally, we introduce a temporal dependence in the learned graph structure
which allows us to incorporate contact measurement updates during execution
thus enabling more accurate forward predictions. The learned stable and locally
linear dynamics enable the use of optimal control algorithms such as iLQR for
long-horizon planning and control for complex interactive tasks. Through
experiments in simulation and in the real world, we evaluate the performance of
our method by using the learned interaction dynamics for control and
demonstrate generalization to more objects and interactions not seen during
training. We introduce a control scheme that takes advantage of contact
measurement updates and hence is robust to prediction inaccuracies during
execution
- …