5 research outputs found
Learning the Solution Operator of Boundary Value Problems using Graph Neural Networks
As an alternative to classical numerical solvers for partial differential
equations (PDEs) subject to boundary value constraints, there has been a surge
of interest in investigating neural networks that can solve such problems
efficiently. In this work, we design a general solution operator for two
different time-independent PDEs using graph neural networks (GNNs) and spectral
graph convolutions. We train the networks on simulated data from a finite
elements solver on a variety of shapes and inhomogeneities. In contrast to
previous works, we focus on the ability of the trained operator to generalize
to previously unseen scenarios. Specifically, we test generalization to meshes
with different shapes and superposition of solutions for a different number of
inhomogeneities. We find that training on a diverse dataset with lots of
variation in the finite element meshes is a key ingredient for achieving good
generalization results in all cases. With this, we believe that GNNs can be
used to learn solution operators that generalize over a range of properties and
produce solutions much faster than a generic solver. Our dataset, which we make
publicly available, can be used and extended to verify the robustness of these
models under varying conditions
Towards Learning Self-Organized Criticality of Rydberg Atoms using Graph Neural Networks
Self-Organized Criticality (SOC) is a ubiquitous dynamical phenomenon
believed to be responsible for the emergence of universal scale-invariant
behavior in many, seemingly unrelated systems, such as forest fires, virus
spreading or atomic excitation dynamics. SOC describes the buildup of
large-scale and long-range spatio-temporal correlations as a result of only
local interactions and dissipation. The simulation of SOC dynamics is typically
based on Monte-Carlo (MC) methods, which are however numerically expensive and
do not scale beyond certain system sizes. We investigate the use of Graph
Neural Networks (GNNs) as an effective surrogate model to learn the dynamics
operator for a paradigmatic SOC system, inspired by an experimentally
accessible physics example: driven Rydberg atoms. To this end, we generalize
existing GNN simulation approaches to predict dynamics for the internal state
of the node. We show that we can accurately reproduce the MC dynamics as well
as generalize along the two important axes of particle number and particle
density. This paves the way to model much larger systems beyond the limits of
traditional MC methods. While the exact system is inspired by the dynamics of
Rydberg atoms, the approach is quite general and can readily be applied to
other systems
Curve Your Enthusiasm: Concurvity Regularization in Differentiable Generalized Additive Models
Generalized Additive Models (GAMs) have recently experienced a resurgence in
popularity due to their interpretability, which arises from expressing the
target value as a sum of non-linear transformations of the features. Despite
the current enthusiasm for GAMs, their susceptibility to concurvity - i.e.,
(possibly non-linear) dependencies between the features - has hitherto been
largely overlooked. Here, we demonstrate how concurvity can severly impair the
interpretability of GAMs and propose a remedy: a conceptually simple, yet
effective regularizer which penalizes pairwise correlations of the non-linearly
transformed feature variables. This procedure is applicable to any
differentiable additive model, such as Neural Additive Models or NeuralProphet,
and enhances interpretability by eliminating ambiguities due to self-canceling
feature contributions. We validate the effectiveness of our regularizer in
experiments on synthetic as well as real-world datasets for time-series and
tabular data. Our experiments show that concurvity in GAMs can be reduced
without significantly compromising prediction quality, improving
interpretability and reducing variance in the feature importances
Auto-Compressing Subset Pruning for Semantic Image Segmentation
State-of-the-art semantic segmentation models are characterized by high
parameter counts and slow inference times, making them unsuitable for
deployment in resource-constrained environments. To address this challenge, we
propose \textsc{Auto-Compressing Subset Pruning}, \acosp, as a new online
compression method. The core of \acosp consists of learning a channel selection
mechanism for individual channels of each convolution in the segmentation model
based on an effective temperature annealing schedule. We show a crucial
interplay between providing a high-capacity model at the beginning of training
and the compression pressure forcing the model to compress concepts into
retained channels. We apply \acosp to \segnet and \pspnet architectures and
show its success when trained on the \camvid, \city, \voc, and \ade datasets.
The results are competitive with existing baselines for compression of
segmentation models at low compression ratios and outperform them significantly
at high compression ratios, yielding acceptable results even when removing more
than of the parameters. In addition, \acosp is conceptually simple, easy
to implement, and can readily be generalized to other data modalities, tasks,
and architectures. Our code is available at
\url{https://github.com/merantix/acosp}.Comment: 10 pages, 5 figures, 1 table, appendi