105 research outputs found
Learning Stable and Robust Linear Parameter-Varying State-Space Models
This paper presents two direct parameterizations of stable and robust linear parameter-varying state-space (LPV-SS) models. The model parametrizations guarantee a priori that for all parameter values during training, the allowed models are stable in the contraction sense or have their Lipschitz constant bounded by a user-defined value γ . Furthermore, since the parametrizations are direct, the models can be trained using unconstrained optimization. The fact that the trained models are of the LPV-SS class makes them useful for, e.g., further convex analysis or controller design. The effectiveness of the approach is demonstrated on an LPV identification problem
Learning Stable and Robust Linear Parameter-Varying State-Space Models
This paper presents two direct parameterizations of stable and robust linear parameter-varying state-space (LPV-SS) models. The model parametrizations guarantee a priori that for all parameter values during training, the allowed models are stable in the contraction sense or have their Lipschitz constant bounded by a user-defined value γ . Furthermore, since the parametrizations are direct, the models can be trained using unconstrained optimization. The fact that the trained models are of the LPV-SS class makes them useful for, e.g., further convex analysis or controller design. The effectiveness of the approach is demonstrated on an LPV identification problem
CubeNet: Equivariance to 3D Rotation and Translation
3D Convolutional Neural Networks are sensitive to transformations applied to
their input. This is a problem because a voxelized version of a 3D object, and
its rotated clone, will look unrelated to each other after passing through to
the last layer of a network. Instead, an idealized model would preserve a
meaningful representation of the voxelized object, while explaining the
pose-difference between the two inputs. An equivariant representation vector
has two components: the invariant identity part, and a discernable encoding of
the transformation. Models that can't explain pose-differences risk "diluting"
the representation, in pursuit of optimizing a classification or regression
loss function.
We introduce a Group Convolutional Neural Network with linear equivariance to
translations and right angle rotations in three dimensions. We call this
network CubeNet, reflecting its cube-like symmetry. By construction, this
network helps preserve a 3D shape's global and local signature, as it is
transformed through successive layers. We apply this network to a variety of 3D
inference problems, achieving state-of-the-art on the ModelNet10 classification
challenge, and comparable performance on the ISBI 2012 Connectome Segmentation
Benchmark. To the best of our knowledge, this is the first 3D rotation
equivariant CNN for voxel representations.Comment: Preprin
Understanding Spectral Graph Neural Network
The graph neural networks have developed by leaps and bounds in recent years
due to the restriction of traditional convolutional filters on non-Euclidean
structured data. Spectral graph theory mainly studies fundamental graph
properties using algebraic methods to analyze the spectrum of the adjacency
matrix of a graph, which lays the foundation of graph convolutional neural
networks. This report is more than notes and self-contained which comes from my
Ph.D. first-year report literature review part, it illustrates how to link
fundamentals of spectral graph theory to graph convolutional neural network
theory, and discusses the major spectral-based graph convolutional neural
networks. The practical applications of the graph neural networks defined in
the spectral domain is also reviewed
Group invariant machine learning by fundamental domain projections
We approach the well-studied problem of supervised group invariant and
equivariant machine learning from the point of view of geometric topology. We
propose a novel approach using a pre-processing step, which involves projecting
the input data into a geometric space which parametrises the orbits of the
symmetry group. This new data can then be the input for an arbitrary machine
learning model (neural network, random forest, support-vector machine etc).
We give an algorithm to compute the geometric projection, which is efficient
to implement, and we illustrate our approach on some example machine learning
problems (including the well-studied problem of predicting Hodge numbers of
CICY matrices), in each case finding an improvement in accuracy versus others
in the literature. The geometric topology viewpoint also allows us to give a
unified description of so-called intrinsic approaches to group equivariant
machine learning, which encompasses many other approaches in the literature.Comment: 21 pages, 4 figure
Exposition on over-squashing problem on GNNs: Current Methods, Benchmarks and Challenges
Graph-based message-passing neural networks (MPNNs) have achieved remarkable
success in both node and graph-level learning tasks. However, several
identified problems, including over-smoothing (OSM), limited expressive power,
and over-squashing (OSQ), still limit the performance of MPNNs. In particular,
OSQ serves as the latest identified problem, where MPNNs gradually lose their
learning accuracy when long-range dependencies between graph nodes are
required. In this work, we provide an exposition on the OSQ problem by
summarizing different formulations of OSQ from current literature, as well as
the three different categories of approaches for addressing the OSQ problem. In
addition, we also discuss the alignment between OSQ and expressive power and
the trade-off between OSQ and OSM. Furthermore, we summarize the empirical
methods leveraged from existing works to verify the efficiency of OSQ
mitigation approaches, with illustrations of their computational complexities.
Lastly, we list some open questions that are of interest for further
exploration of the OSQ problem along with potential directions from the best of
our knowledge
- …