298 research outputs found
Time-changed normalizing flows for accurate SDE modeling
The generative paradigm has become increasingly important in machine learning
and deep learning models. Among popular generative models are normalizing
flows, which enable exact likelihood estimation by transforming a base
distribution through diffeomorphic transformations. Extending the normalizing
flow framework to handle time-indexed flows gave dynamic normalizing flows, a
powerful tool to model time series, stochastic processes, and neural stochastic
differential equations (SDEs). In this work, we propose a novel variant of
dynamic normalizing flows, a Time Changed Normalizing Flow (TCNF), based on
time deformation of a Brownian motion which constitutes a versatile and
extensive family of Gaussian processes. This approach enables us to effectively
model some SDEs, that cannot be modeled otherwise, including standard ones such
as the well-known Ornstein-Uhlenbeck process, and generalizes prior
methodologies, leading to improved results and better inference and prediction
capability
Diffeomorphic Transformations for Time Series Analysis: An Efficient Approach to Nonlinear Warping
The proliferation and ubiquity of temporal data across many disciplines has
sparked interest for similarity, classification and clustering methods
specifically designed to handle time series data. A core issue when dealing
with time series is determining their pairwise similarity, i.e., the degree to
which a given time series resembles another. Traditional distance measures such
as the Euclidean are not well-suited due to the time-dependent nature of the
data. Elastic metrics such as dynamic time warping (DTW) offer a promising
approach, but are limited by their computational complexity,
non-differentiability and sensitivity to noise and outliers. This thesis
proposes novel elastic alignment methods that use parametric \& diffeomorphic
warping transformations as a means of overcoming the shortcomings of DTW-based
metrics. The proposed method is differentiable \& invertible, well-suited for
deep learning architectures, robust to noise and outliers, computationally
efficient, and is expressive and flexible enough to capture complex patterns.
Furthermore, a closed-form solution was developed for the gradient of these
diffeomorphic transformations, which allows an efficient search in the
parameter space, leading to better solutions at convergence. Leveraging the
benefits of these closed-form diffeomorphic transformations, this thesis
proposes a suite of advancements that include: (a) an enhanced temporal
transformer network for time series alignment and averaging, (b) a
deep-learning based time series classification model to simultaneously align
and classify signals with high accuracy, (c) an incremental time series
clustering algorithm that is warping-invariant, scalable and can operate under
limited computational and time resources, and finally, (d) a normalizing flow
model that enhances the flexibility of affine transformations in coupling and
autoregressive layers.Comment: PhD Thesis, defended at the University of Navarra on July 17, 2023.
277 pages, 8 chapters, 1 appendi
Bayesian Estimation of White Matter Atlas from High Angular Resolution Diffusion Imaging
We present a Bayesian probabilistic model to estimate the brain white matter
atlas from high angular resolution diffusion imaging (HARDI) data. This model
incorporates a shape prior of the white matter anatomy and the likelihood of
individual observed HARDI datasets. We first assume that the atlas is generated
from a known hyperatlas through a flow of diffeomorphisms and its shape prior
can be constructed based on the framework of large deformation diffeomorphic
metric mapping (LDDMM). LDDMM characterizes a nonlinear diffeomorphic shape
space in a linear space of initial momentum uniquely determining diffeomorphic
geodesic flows from the hyperatlas. Therefore, the shape prior of the HARDI
atlas can be modeled using a centered Gaussian random field (GRF) model of the
initial momentum. In order to construct the likelihood of observed HARDI
datasets, it is necessary to study the diffeomorphic transformation of
individual observations relative to the atlas and the probabilistic
distribution of orientation distribution functions (ODFs). To this end, we
construct the likelihood related to the transformation using the same
construction as discussed for the shape prior of the atlas. The probabilistic
distribution of ODFs is then constructed based on the ODF Riemannian manifold.
We assume that the observed ODFs are generated by an exponential map of random
tangent vectors at the deformed atlas ODF. Hence, the likelihood of the ODFs
can be modeled using a GRF of their tangent vectors in the ODF Riemannian
manifold. We solve for the maximum a posteriori using the
Expectation-Maximization algorithm and derive the corresponding update
equations. Finally, we illustrate the HARDI atlas constructed based on a
Chinese aging cohort of 94 adults and compare it with that generated by
averaging the coefficients of spherical harmonics of the ODF across subjects
Neural Diffeomorphic Non-uniform B-spline Flows
Normalizing flows have been successfully modeling a complex probability
distribution as an invertible transformation of a simple base distribution.
However, there are often applications that require more than invertibility. For
instance, the computation of energies and forces in physics requires the second
derivatives of the transformation to be well-defined and continuous. Smooth
normalizing flows employ infinitely differentiable transformation, but with the
price of slow non-analytic inverse transforms. In this work, we propose
diffeomorphic non-uniform B-spline flows that are at least twice continuously
differentiable while bi-Lipschitz continuous, enabling efficient
parametrization while retaining analytic inverse transforms based on a
sufficient condition for diffeomorphism. Firstly, we investigate the sufficient
condition for Ck-2-diffeomorphic non-uniform kth-order B-spline
transformations. Then, we derive an analytic inverse transformation of the
non-uniform cubic B-spline transformation for neural diffeomorphic non-uniform
B-spline flows. Lastly, we performed experiments on solving the force matching
problem in Boltzmann generators, demonstrating that our C2-diffeomorphic
non-uniform B-spline flows yielded solutions better than previous spline flows
and faster than smooth normalizing flows. Our source code is publicly available
at https://github.com/smhongok/Non-uniform-B-spline-Flow.Comment: Accepted to AAAI 202
Semi-supervised Learning of Pushforwards For Domain Translation & Adaptation
Given two probability densities on related data spaces, we seek a map pushing
one density to the other while satisfying application-dependent constraints.
For maps to have utility in a broad application space (including domain
translation, domain adaptation, and generative modeling), the map must be
available to apply on out-of-sample data points and should correspond to a
probabilistic model over the two spaces. Unfortunately, existing approaches,
which are primarily based on optimal transport, do not address these needs. In
this paper, we introduce a novel pushforward map learning algorithm that
utilizes normalizing flows to parameterize the map. We first re-formulate the
classical optimal transport problem to be map-focused and propose a learning
algorithm to select from all possible maps under the constraint that the map
minimizes a probability distance and application-specific regularizers; thus,
our method can be seen as solving a modified optimal transport problem. Once
the map is learned, it can be used to map samples from a source domain to a
target domain. In addition, because the map is parameterized as a composition
of normalizing flows, it models the empirical distributions over the two data
spaces and allows both sampling and likelihood evaluation for both data sets.
We compare our method (parOT) to related optimal transport approaches in the
context of domain adaptation and domain translation on benchmark data sets.
Finally, to illustrate the impact of our work on applied problems, we apply
parOT to a real scientific application: spectral calibration for
high-dimensional measurements from two vastly different environmentsComment: 19 pages, 7 figure
- …