Blind source separation of tensor-valued time series

The blind source separation model for multivariate time series generally assumes that the observed series is a linear transformation of an unobserved series with temporally uncorrelated or independent components. Given the observations, the objective is to find a linear transformation that recovers the latent series. Several methods for accomplishing this exist and three particular ones are the classic SOBI and the recently proposed generalized FOBI (gFOBI) and generalized JADE (gJADE), each based on the use of joint lagged moments. In this paper we generalize the methodologies behind these algorithms for tensor-valued time series. We assume that our data consists of a tensor observed at each time point and that the observations are linear transformations of latent tensors we wish to estimate. The tensorial generalizations are shown to have particularly elegant forms and we show that each of them is Fisher consistent and orthogonal equivariant. Comparing the new methods with the original ones in various settings shows that the tensorial extensions are superior to both their vector-valued counterparts and to two existing tensorial dimension reduction methods for i.i.d. data. Finally, applications to fMRI-data and video processing show that the methods are capable of extracting relevant information from noisy high-dimensional data.Comment: 26 pages, 6 figure

Multi-Scale Factor Analysis of High-Dimensional Brain Signals

In this paper, we develop an approach to modeling high-dimensional networks with a large number of nodes arranged in a hierarchical and modular structure. We propose a novel multi-scale factor analysis (MSFA) model which partitions the massive spatio-temporal data defined over the complex networks into a finite set of regional clusters. To achieve further dimension reduction, we represent the signals in each cluster by a small number of latent factors. The correlation matrix for all nodes in the network are approximated by lower-dimensional sub-structures derived from the cluster-specific factors. To estimate regional connectivity between numerous nodes (within each cluster), we apply principal components analysis (PCA) to produce factors which are derived as the optimal reconstruction of the observed signals under the squared loss. Then, we estimate global connectivity (between clusters or sub-networks) based on the factors across regions using the RV-coefficient as the cross-dependence measure. This gives a reliable and computationally efficient multi-scale analysis of both regional and global dependencies of the large networks. The proposed novel approach is applied to estimate brain connectivity networks using functional magnetic resonance imaging (fMRI) data. Results on resting-state fMRI reveal interesting modular and hierarchical organization of human brain networks during rest.Comment: 43 page

Bayesian Mixed Effect Sparse Tensor Response Regression Model with Joint Estimation of Activation and Connectivity

Brain activation and connectivity analyses in task-based functional magnetic resonance imaging (fMRI) experiments with multiple subjects are currently at the forefront of data-driven neuroscience. In such experiments, interest often lies in understanding activation of brain voxels due to external stimuli and strong association or connectivity between the measurements on a set of pre-specified group of brain voxels, also known as regions of interest (ROI). This article proposes a joint Bayesian additive mixed modeling framework that simultaneously assesses brain activation and connectivity patterns from multiple subjects. In particular, fMRI measurements from each individual obtained in the form of a multi-dimensional array/tensor at each time are regressed on functions of the stimuli. We impose a low-rank PARAFAC decomposition on the tensor regression coefficients corresponding to the stimuli to achieve parsimony. Multiway stick breaking shrinkage priors are employed to infer activation patterns and associated uncertainties in each voxel. Further, the model introduces region specific random effects which are jointly modeled with a Bayesian Gaussian graphical prior to account for the connectivity among pairs of ROIs. Empirical investigations under various simulation studies demonstrate the effectiveness of the method as a tool to simultaneously assess brain activation and connectivity. The method is then applied to a multi-subject fMRI dataset from a balloon-analog risk-taking experiment in order to make inference about how the brain processes risk.Comment: 27 pages, 7 figure

Linked Component Analysis from Matrices to High Order Tensors: Applications to Biomedical Data

With the increasing availability of various sensor technologies, we now have access to large amounts of multi-block (also called multi-set, multi-relational, or multi-view) data that need to be jointly analyzed to explore their latent connections. Various component analysis methods have played an increasingly important role for the analysis of such coupled data. In this paper, we first provide a brief review of existing matrix-based (two-way) component analysis methods for the joint analysis of such data with a focus on biomedical applications. Then, we discuss their important extensions and generalization to multi-block multiway (tensor) data. We show how constrained multi-block tensor decomposition methods are able to extract similar or statistically dependent common features that are shared by all blocks, by incorporating the multiway nature of data. Special emphasis is given to the flexible common and individual feature analysis of multi-block data with the aim to simultaneously extract common and individual latent components with desired properties and types of diversity. Illustrative examples are given to demonstrate their effectiveness for biomedical data analysis.Comment: 20 pages, 11 figures, Proceedings of the IEEE, 201

Large Spectral Density Matrix Estimation by Thresholding

Spectral density matrix estimation of multivariate time series is a classical problem in time series and signal processing. In modern neuroscience, spectral density based metrics are commonly used for analyzing functional connectivity among brain regions. In this paper, we develop a non-asymptotic theory for regularized estimation of high-dimensional spectral density matrices of Gaussian and linear processes using thresholded versions of averaged periodograms. Our theoretical analysis ensures that consistent estimation of spectral density matrix of a $p$-dimensional time series using $n$ samples is possible under high-dimensional regime $\log p / n \rightarrow 0$ as long as the true spectral density is approximately sparse. A key technical component of our analysis is a new concentration inequality of average periodogram around its expectation, which is of independent interest. Our estimation consistency results complement existing results for shrinkage based estimators of multivariate spectral density, which require no assumption on sparsity but only ensure consistent estimation in a regime $p^2/n \rightarrow 0$. In addition, our proposed thresholding based estimators perform consistent and automatic edge selection when learning coherence networks among the components of a multivariate time series. We demonstrate the advantage of our estimators using simulation studies and a real data application on functional connectivity analysis with fMRI data

Variational Mixture Models with Gamma or inverse-Gamma components

Mixture models with Gamma and or inverse-Gamma distributed mixture components are useful for medical image tissue segmentation or as post-hoc models for regression coefficients obtained from linear regression within a Generalised Linear Modeling framework (GLM), used in this case to separate stochastic (Gaussian) noise from some kind of positive or negative "activation" (modeled as Gamma or inverse-Gamma distributed). To date, the most common choice in this context it is Gaussian/Gamma mixture models learned through a maximum likelihood (ML) approach; we recently extended such algorithm for mixture models with inverse-Gamma components. Here, we introduce a fully analytical Variational Bayes (VB) learning framework for both Gamma and/or inverse-Gamma components. We use synthetic and resting state fMRI data to compare the performance of the ML and VB algorithms in terms of area under the curve and computational cost. We observed that the ML Gaussian/Gamma model is very expensive specially when considering high resolution images; furthermore, these solutions are highly variable and they occasionally can overestimate the activations severely. The Bayesian Gauss-Gamma is in general the fastest algorithm but provides too dense solutions. The maximum likelihood Gaussian/inverse-Gamma is also very fast but provides in general very sparse solutions. The variational Gaussian/inverse-Gamma mixture model is the most robust and its cost is acceptable even for high resolution images. Further, the presented methodology represents an essential building block that can be directly used in more complex inference tasks, specially designed to analyse MRI-fMRI data; such models include for example analytical variational mixture models with adaptive spatial regularization or better source models for new spatial blind source separation approaches

Decoding the Encoding of Functional Brain Networks: an fMRI Classification Comparison of Non-negative Matrix Factorization (NMF), Independent Component Analysis (ICA), and Sparse Coding Algorithms

Brain networks in fMRI are typically identified using spatial independent component analysis (ICA), yet mathematical constraints such as sparse coding and positivity both provide alternate biologically-plausible frameworks for generating brain networks. Non-negative Matrix Factorization (NMF) would suppress negative BOLD signal by enforcing positivity. Spatial sparse coding algorithms ($L1$ Regularized Learning and K-SVD) would impose local specialization and a discouragement of multitasking, where the total observed activity in a single voxel originates from a restricted number of possible brain networks. The assumptions of independence, positivity, and sparsity to encode task-related brain networks are compared; the resulting brain networks for different constraints are used as basis functions to encode the observed functional activity at a given time point. These encodings are decoded using machine learning to compare both the algorithms and their assumptions, using the time series weights to predict whether a subject is viewing a video, listening to an audio cue, or at rest, in 304 fMRI scans from 51 subjects. For classifying cognitive activity, the sparse coding algorithm of $L1$ Regularized Learning consistently outperformed 4 variations of ICA across different numbers of networks and noise levels (p$<$0.001). The NMF algorithms, which suppressed negative BOLD signal, had the poorest accuracy. Within each algorithm, encodings using sparser spatial networks (containing more zero-valued voxels) had higher classification accuracy (p$<$0.001). The success of sparse coding algorithms may suggest that algorithms which enforce sparse coding, discourage multitasking, and promote local specialization may capture better the underlying source processes than those which allow inexhaustible local processes such as ICA

A Journey from Improper Gaussian Signaling to Asymmetric Signaling

The deviation of continuous and discrete complex random variables from the traditional proper and symmetric assumption to a generalized improper and asymmetric characterization (accounting correlation between a random entity and its complex conjugate), respectively, introduces new design freedom and various potential merits. As such, the theory of impropriety has vast applications in medicine, geology, acoustics, optics, image and pattern recognition, computer vision, and other numerous research fields with our main focus on the communication systems. The journey begins from the design of improper Gaussian signaling in the interference-limited communications and leads to a more elaborate and practically feasible asymmetric discrete modulation design. Such asymmetric shaping bridges the gap between theoretically and practically achievable limits with sophisticated transceiver and detection schemes in both coded/uncoded wireless/optical communication systems. Interestingly, introducing asymmetry and adjusting the transmission parameters according to some design criterion render optimal performance without affecting the bandwidth or power requirements of the systems. This dual-flavored article initially presents the tutorial base content covering the interplay of reality/complexity, propriety/impropriety and circularity/noncircularity and then surveys majority of the contributions in this enormous journey.Comment: IEEE COMST (Early Access

Thresholded Multiscale Gaussian Processes with Application to Bayesian Feature Selection for Massive Neuroimaging Data

Motivated by the needs of selecting important features for massive neuroimaging data, we propose a spatially varying coefficient model (SVCMs) with sparsity and piecewise smoothness imposed on the coefficient functions. A new class of nonparametric priors is developed based on thresholded multiresolution Gaussian processes (TMGP). We show that the TMGP has a large support on a space of sparse and piecewise smooth functions, leading to posterior consistency in coefficient function estimation and feature selection. Also, we develop a method for prior specifications of thresholding parameters in TMGPs and discuss their theoretical properties. Efficient posterior computation algorithms are developed by adopting a kernel convolution approach, where a modified square exponential kernel is chosen taking the advantage that the analytical form of the eigen decomposition is available. Based on simulation studies, we demonstrate that our methods can achieve better performance in estimating the spatially varying coefficient. Also, the proposed model has been applied to an analysis of resting state functional magnetic resonance imaging (Rs-fMRI) data from the Autism Brain Imaging Data Exchange (ABIDE) study, it provides biologically meaningful results.Comment: 37 pages, 7 figure

Determining the Dimension of the Improper Signal Subspace in Complex-Valued Data

A complex-valued signal is improper if it is correlated with its complex conjugate. The dimension of the improper signal subspace, i.e., the number of improper components in a complex-valued measurement, is an important parameter and is unknown in most applications. In this letter, we introduce two approaches to estimate this dimension, one based on an information- theoretic criterion and one based on hypothesis testing. We also present reduced-rank versions of these approaches that work for scenarios where the number of observations is comparable to or even smaller than the dimension of the data. Unlike other techniques for determining model orders, our techniques also work in the presence of additive colored noise