391 research outputs found
A simple and objective method for reproducible resting state network (RSN) detection in fMRI
Spatial Independent Component Analysis (ICA) decomposes the time by space
functional MRI (fMRI) matrix into a set of 1-D basis time courses and their
associated 3-D spatial maps that are optimized for mutual independence. When
applied to resting state fMRI (rsfMRI), ICA produces several spatial
independent components (ICs) that seem to have biological relevance - the
so-called resting state networks (RSNs). The ICA problem is well posed when the
true data generating process follows a linear mixture of ICs model in terms of
the identifiability of the mixing matrix. However, the contrast function used
for promoting mutual independence in ICA is dependent on the finite amount of
observed data and is potentially non-convex with multiple local minima. Hence,
each run of ICA could produce potentially different IC estimates even for the
same data. One technique to deal with this run-to-run variability of ICA was
proposed by Yang et al. (2008) in their algorithm RAICAR which allows for the
selection of only those ICs that have a high run-to-run reproducibility. We
propose an enhancement to the original RAICAR algorithm that enables us to
assign reproducibility p-values to each IC and allows for an objective
assessment of both within subject and across subjects reproducibility. We call
the resulting algorithm RAICAR-N (N stands for null hypothesis test), and we
have applied it to publicly available human rsfMRI data (http://www.nitrc.org).
Our reproducibility analyses indicated that many of the published RSNs in
rsfMRI literature are highly reproducible. However, we found several other RSNs
that are highly reproducible but not frequently listed in the literature.Comment: 54 pages, 13 figure
Constrained Overcomplete Analysis Operator Learning for Cosparse Signal Modelling
We consider the problem of learning a low-dimensional signal model from a
collection of training samples. The mainstream approach would be to learn an
overcomplete dictionary to provide good approximations of the training samples
using sparse synthesis coefficients. This famous sparse model has a less well
known counterpart, in analysis form, called the cosparse analysis model. In
this new model, signals are characterised by their parsimony in a transformed
domain using an overcomplete (linear) analysis operator. We propose to learn an
analysis operator from a training corpus using a constrained optimisation
framework based on L1 optimisation. The reason for introducing a constraint in
the optimisation framework is to exclude trivial solutions. Although there is
no final answer here for which constraint is the most relevant constraint, we
investigate some conventional constraints in the model adaptation field and use
the uniformly normalised tight frame (UNTF) for this purpose. We then derive a
practical learning algorithm, based on projected subgradients and
Douglas-Rachford splitting technique, and demonstrate its ability to robustly
recover a ground truth analysis operator, when provided with a clean training
set, of sufficient size. We also find an analysis operator for images, using
some noisy cosparse signals, which is indeed a more realistic experiment. As
the derived optimisation problem is not a convex program, we often find a local
minimum using such variational methods. Some local optimality conditions are
derived for two different settings, providing preliminary theoretical support
for the well-posedness of the learning problem under appropriate conditions.Comment: 29 pages, 13 figures, accepted to be published in TS
Advances in identifiability of nonlinear probabilistic models
Identifiability is a highly prized property of statistical models. This thesis investigates this property in nonlinear models encountered in two fields of statistics: representation learning and causal discovery. In representation learning, identifiability leads to learning interpretable and reproducible representations, while in causal discovery, it is necessary for the estimation of correct causal directions.
We begin by leveraging recent advances in nonlinear ICA to show that the latent space of a VAE is identifiable up to a permutation and pointwise nonlinear transformations of its components. A factorized prior distribution over the latent variables conditioned on an auxiliary observed variable, such as a class label or nearly any other observation, is required for our result. We also extend previous identifiability results in nonlinear ICA to the case of noisy or undercomplete observations, and incorporate them into a maximum likelihood framework.
Our second contribution is to develop the Independently Modulated Component Analysis (IMCA) framework, a generalization of nonlinear ICA to non-independent latent variables. We show that we can drop the independence assumption in ICA while maintaining identifiability, resulting in a very flexible and generic framework for principled disentangled representation learning. This finding is predicated on the existence of an auxiliary variable that modulates the joint distribution of the latent variables in a factorizable manner.
As a third contribution, we extend the identifiability theory to a broad family of conditional energy-based models (EBMs). This novel model generalizes earlier results by removing any distributional assumptions on the representations, which are ubiquitous in the latent variable setting. The conditional EBM can learn identifiable overcomplete representations and has universal approximation capabilities/.
Finally, we investigate a connection between the framework of autoregressive normalizing flow models and causal discovery. Causal models derived from affine autoregressive flows are shown to be identifiable, generalizing the wellknown additive noise model. Using normalizing flows, we can compute the exact likelihood of the causal model, which is subsequently used to derive a likelihood ratio measure for causal discovery. They are also invertible, making them perfectly suitable for performing causal inference tasks like interventions and counterfactuals
LOCUS: A Novel Decomposition Method for Brain Network Connectivity Matrices using Low-rank Structure with Uniform Sparsity
Network-oriented research has been increasingly popular in many scientific
areas. In neuroscience research, imaging-based network connectivity measures
have become the key for understanding brain organizations, potentially serving
as individual neural fingerprints. There are major challenges in analyzing
connectivity matrices including the high dimensionality of brain networks,
unknown latent sources underlying the observed connectivity, and the large
number of brain connections leading to spurious findings. In this paper, we
propose a novel blind source separation method with low-rank structure and
uniform sparsity (LOCUS) as a fully data-driven decomposition method for
network measures. Compared with the existing method that vectorizes
connectivity matrices ignoring brain network topology, LOCUS achieves more
efficient and accurate source separation for connectivity matrices using
low-rank structure. We propose a novel angle-based uniform sparsity
regularization that demonstrates better performance than the existing sparsity
controls for low-rank tensor methods. We propose a highly efficient iterative
Node-Rotation algorithm that exploits the block multi-convexity of the
objective function to solve the non-convex optimization problem for learning
LOCUS. We illustrate the advantage of LOCUS through extensive simulation
studies. Application of LOCUS to Philadelphia Neurodevelopmental Cohort
neuroimaging study reveals biologically insightful connectivity traits which
are not found using the existing method
Learning World Models with Identifiable Factorization
Extracting a stable and compact representation of the environment is crucial
for efficient reinforcement learning in high-dimensional, noisy, and
non-stationary environments. Different categories of information coexist in
such environments -- how to effectively extract and disentangle these
information remains a challenging problem. In this paper, we propose IFactor, a
general framework to model four distinct categories of latent state variables
that capture various aspects of information within the RL system, based on
their interactions with actions and rewards. Our analysis establishes
block-wise identifiability of these latent variables, which not only provides a
stable and compact representation but also discloses that all reward-relevant
factors are significant for policy learning. We further present a practical
approach to learning the world model with identifiable blocks, ensuring the
removal of redundants but retaining minimal and sufficient information for
policy optimization. Experiments in synthetic worlds demonstrate that our
method accurately identifies the ground-truth latent variables, substantiating
our theoretical findings. Moreover, experiments in variants of the DeepMind
Control Suite and RoboDesk showcase the superior performance of our approach
over baselines
A stochastic algorithm for probabilistic independent component analysis
The decomposition of a sample of images on a relevant subspace is a recurrent
problem in many different fields from Computer Vision to medical image
analysis. We propose in this paper a new learning principle and implementation
of the generative decomposition model generally known as noisy ICA (for
independent component analysis) based on the SAEM algorithm, which is a
versatile stochastic approximation of the standard EM algorithm. We demonstrate
the applicability of the method on a large range of decomposition models and
illustrate the developments with experimental results on various data sets.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS499 the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
- …