15 research outputs found
Consistent Dynamic Mode Decomposition
We propose a new method for computing Dynamic Mode Decomposition (DMD)
evolution matrices, which we use to analyze dynamical systems. Unlike the
majority of existing methods, our approach is based on a variational
formulation consisting of data alignment penalty terms and constitutive
orthogonality constraints. Our method does not make any assumptions on the
structure of the data or their size, and thus it is applicable to a wide range
of problems including non-linear scenarios or extremely small observation sets.
In addition, our technique is robust to noise that is independent of the
dynamics and it does not require input data to be sequential. Our key idea is
to introduce a regularization term for the forward and backward dynamics. The
obtained minimization problem is solved efficiently using the Alternating
Method of Multipliers (ADMM) which requires two Sylvester equation solves per
iteration. Our numerical scheme converges empirically and is similar to a
provably convergent ADMM scheme. We compare our approach to various
state-of-the-art methods on several benchmark dynamical systems
A Koopman Approach to Understanding Sequence Neural Models
We introduce a new approach to understanding trained sequence neural models:
the Koopman Analysis of Neural Networks (KANN) method. Motivated by the
relation between time-series models and self-maps, we compute approximate
Koopman operators that encode well the latent dynamics. Unlike other existing
methods whose applicability is limited, our framework is global, and it has
only weak constraints over the inputs. Moreover, the Koopman operator is
linear, and it is related to a rich mathematical theory. Thus, we can use tools
and insights from linear analysis and Koopman Theory in our study. For
instance, we show that the operator eigendecomposition is instrumental in
exploring the dominant features of the network. Our results extend across tasks
and architectures as we demonstrate for the copy problem, and ECG
classification and sentiment analysis tasks
Generative Modeling of Graphs via Joint Diffusion of Node and Edge Attributes
Graph generation is integral to various engineering and scientific
disciplines. Nevertheless, existing methodologies tend to overlook the
generation of edge attributes. However, we identify critical applications where
edge attributes are essential, making prior methods potentially unsuitable in
such contexts. Moreover, while trivial adaptations are available, empirical
investigations reveal their limited efficacy as they do not properly model the
interplay among graph components. To address this, we propose a joint
score-based model of nodes and edges for graph generation that considers all
graph components. Our approach offers two key novelties: (i) node and edge
attributes are combined in an attention module that generates samples based on
the two ingredients; and (ii) node, edge and adjacency information are mutually
dependent during the graph diffusion process. We evaluate our method on
challenging benchmarks involving real-world and synthetic datasets in which
edge features are crucial. Additionally, we introduce a new synthetic dataset
that incorporates edge values. Furthermore, we propose a novel application that
greatly benefits from the method due to its nature: the generation of traffic
scenes represented as graphs. Our method outperforms other graph generation
methods, demonstrating a significant advantage in edge-related measures
Eigenvalue initialisation and regularisation for Koopman autoencoders
Regularising the parameter matrices of neural networks is ubiquitous in
training deep models. Typical regularisation approaches suggest initialising
weights using small random values, and to penalise weights to promote sparsity.
However, these widely used techniques may be less effective in certain
scenarios. Here, we study the Koopman autoencoder model which includes an
encoder, a Koopman operator layer, and a decoder. These models have been
designed and dedicated to tackle physics-related problems with interpretable
dynamics and an ability to incorporate physics-related constraints. However,
the majority of existing work employs standard regularisation practices. In our
work, we take a step toward augmenting Koopman autoencoders with initialisation
and penalty schemes tailored for physics-related settings. Specifically, we
propose the "eigeninit" initialisation scheme that samples initial Koopman
operators from specific eigenvalue distributions. In addition, we suggest the
"eigenloss" penalty scheme that penalises the eigenvalues of the Koopman
operator during training. We demonstrate the utility of these schemes on two
synthetic data sets: a driven pendulum and flow past a cylinder; and two
real-world problems: ocean surface temperatures and cyclone wind fields. We
find on these datasets that eigenloss and eigeninit improves the convergence
rate by up to a factor of 5, and that they reduce the cumulative long-term
prediction error by up to a factor of 3. Such a finding points to the utility
of incorporating similar schemes as an inductive bias in other physics-related
deep learning approaches.Comment: 18 page
Consistent functional cross field design for mesh quadrangulation
International audienceWe propose a novel technique for computing consistent cross fields on a pair of triangle meshes given an input correspondence, which we use as guiding fields for approximately consistent quadrangulations. Unlike the majority of existing methods our approach does not assume that the meshes share the same connectivity or even have the same number of vertices, and furthermore does not place any restrictions on the topology (genus) of the shapes. Importantly, our method is robust with respect to small perturbations of the given correspondence, as it only relies on the transportation of real-valued functions and thus avoids the costly and error-prone estimation of the map differential. Key to this robustness is a novel formulation, which relies on the previously-proposed notion of power vectors, and we show how consistency can be enforced without pre-alignment of local basis frames, in which these power vectors are computed. We demonstrate that using the same formulation we can both compute a quadrangulation that would respect a given symmetry on the same shape or a map across a pair of shapes. We provide quantitative and qualitative comparison of our method with several baselines and show that it both provides more accurate results and allows to handle more general cases than existing techniques