Paradigm free mapping: detection and characterization of single trial fMRI BOLD responses without prior stimulus information

The increased contrast to noise ratio available at Ultrahigh (7T) Magnetic Resonance Imaging (MRI) allows mapping in space and time the brain's response to single trial events with functional MRI (fMRI) based on the Blood Oxygenation Level Dependent (BOLD) contrast. This thesis primarily concerns with the development of techniques to detect and characterize single trial event-related BOLD responses without prior paradigm information, Paradigm Free Mapping, and assess variations in BOLD sensitivity across brain regions at high field fMRI. Based on a linear haemodynamic response model, Paradigm Free Mapping (PFM) techniques rely on the deconvolution of the neuronal-related signal driving the BOLD effect using regularized least squares estimators. The first approach, named PFM, builds on the ridge regression estimator and spatio-temporal t-statistics to detect statistically significant changes in the deconvolved fMRI signal. The second method, Sparse PFM, benefits from subset selection features of the LASSO and Dantzig Selector estimators that automatically detect the single trial BOLD responses by promoting a sparse deconvolution of the signal. The third technique, Multicomponent PFM, exploits further the benefits of sparse estimation to decompose the fMRI signal into a haemodynamical component and a baseline component using the morphological component analysis algorithm. These techniques were evaluated in simulations and experimental fMRI datasets, and the results were compared with well-established fMRI analysis methods. In particular, the methods developed here enabled the detection of single trial BOLD responses to visually-cued and self-paced finger tapping responses without prior information of the events. The potential application of Sparse PFM to identify interictal discharges in idiopathic generalized epilepsy was also investigated. Furthermore, Multicomponent PFM allowed us to extract cardiac and respiratory fluctuations of the signal without the need of physiological monitoring. To sum up, this work demonstrates the feasibility to do single trial fMRI analysis without prior stimulus or physiological information using PFM techniques

Computational and Statistical Aspects of High-Dimensional Structured Estimation

University of Minnesota Ph.D. dissertation. May 2018. Major: Computer Science. Advisor: Arindam Banerjee. 1 computer file (PDF); xiii, 256 pages.Modern statistical learning often faces high-dimensional data, for which the number of features that should be considered is very large. In consideration of various constraints encountered in data collection, such as cost and time, however, the available samples for applications in certain domains are of small size compared with the feature sets. In this scenario, statistical estimation becomes much more challenging than in the large-sample regime. Since the information revealed by small samples is inadequate for finding the optimal model parameters, the estimator may end up with incorrect models that appear to fit the observed data but fail to generalize to unseen ones. Owning to the prior knowledge about the underlying parameters, additional structures can be imposed to effectively reduce the parameter space, in which it is easier to identify the true one with limited data. This simple idea has inspired the study of high-dimensional statistics since its inception. Over the last two decades, sparsity has been one of the most popular structures to exploit when we estimate a high-dimensional parameter, which assumes that the number of nonzero elements in parameter vector/matrix is much smaller than its ambient dimension. For simple scenarios such as linear models, L1-norm based convex estimators like Lasso and Dantzig selector, have been widely used to find the true parameter with reasonable amount of computation and provably small error. Recent years have also seen a variety of structures proposed beyond sparsity, e.g., group sparsity and low-rankness of matrix, which are demonstrated to be useful in many applications. On the other hand, the aforementioned estimators can be extended to leverage new types of structures by finding appropriate convex surrogates like the L1 norm for sparsity. Despite their success on individual structures, current developments towards a unified understanding of various structures are still incomplete in both computational and statistical aspects. Moreover, due to the nature of the model or the parameter structure, the associated estimator can be inherently non-convex, which may need additional care when we consider such unification of different structures. In this thesis, we aim to make progress towards a unified framework for the estimation with general structures, by studying the high-dimensional structured linear model and other semi-parametric and non-convex extensions. In particular, we introduce the generalized Dantzig selector (GDS), which extends the original Dantzig selector for sparse linear models. For the computational aspect, we develop an efficient optimization algorithm to compute the GDS. On statistical side, we establish the recovery guarantees of GDS using certain geometric measures. Then we demonstrate that those geometric measures can be bounded by utilizing simple information of the structures. These results on GDS have been extended to the matrix setting as well. Apart from the linear model, we also investigate one of its semi-parametric extension -- the single-index model (SIM). To estimate the true parameter, we incorporate its structure into two types of simple estimators, whose estimation error can be established using similar geometric measures. Besides we also design a new semi-parametric model called sparse linear isotonic model (SLIM), for which we provide an efficient estimation algorithm along with its statistical guarantees. Lastly, we consider the non-convex estimation for structured multi-response linear models. We propose an alternating estimation procedure to estimate the parameters. In spite of dealing with non-convexity, we show that the statistical guarantees for general structures can be also summarized by the geometric measures

Paradigm free mapping: detection and characterization of single trial fMRI BOLD responses without prior stimulus information

The increased contrast to noise ratio available at Ultrahigh (7T) Magnetic Resonance Imaging (MRI) allows mapping in space and time the brain's response to single trial events with functional MRI (fMRI) based on the Blood Oxygenation Level Dependent (BOLD) contrast. This thesis primarily concerns with the development of techniques to detect and characterize single trial event-related BOLD responses without prior paradigm information, Paradigm Free Mapping, and assess variations in BOLD sensitivity across brain regions at high field fMRI. Based on a linear haemodynamic response model, Paradigm Free Mapping (PFM) techniques rely on the deconvolution of the neuronal-related signal driving the BOLD effect using regularized least squares estimators. The first approach, named PFM, builds on the ridge regression estimator and spatio-temporal t-statistics to detect statistically significant changes in the deconvolved fMRI signal. The second method, Sparse PFM, benefits from subset selection features of the LASSO and Dantzig Selector estimators that automatically detect the single trial BOLD responses by promoting a sparse deconvolution of the signal. The third technique, Multicomponent PFM, exploits further the benefits of sparse estimation to decompose the fMRI signal into a haemodynamical component and a baseline component using the morphological component analysis algorithm. These techniques were evaluated in simulations and experimental fMRI datasets, and the results were compared with well-established fMRI analysis methods. In particular, the methods developed here enabled the detection of single trial BOLD responses to visually-cued and self-paced finger tapping responses without prior information of the events. The potential application of Sparse PFM to identify interictal discharges in idiopathic generalized epilepsy was also investigated. Furthermore, Multicomponent PFM allowed us to extract cardiac and respiratory fluctuations of the signal without the need of physiological monitoring. To sum up, this work demonstrates the feasibility to do single trial fMRI analysis without prior stimulus or physiological information using PFM techniques

Statistical methods for the testing and estimation of linear dependence structures on paired high-dimensional data: application to genomic data

This thesis provides novel methodology for statistical analysis of paired high-dimensional genomic data, with the aimto identify gene interactions specific to each group of samples as well as the gene connections that change between the two classes of observations. An example of such groups can be patients under two medical conditions, in which the estimation of gene interaction networks is relevant to biologists as part of discerning gene regulatory mechanisms that control a disease process like, for instance, cancer. We construct these interaction networks fromdata by considering the non-zero structure of correlationmatrices, which measure linear dependence between random variables, and their inversematrices, which are commonly known as precision matrices and determine linear conditional dependence instead. In this regard, we study three statistical problems related to the testing, single estimation and joint estimation of (conditional) dependence structures. Firstly, we develop hypothesis testingmethods to assess the equality of two correlation matrices, and also two correlation sub-matrices, corresponding to two classes of samples, and hence the equality of the underlying gene interaction networks. We consider statistics based on the average of squares, maximum and sum of exceedances of sample correlations, which are suitable for both independent and paired observations. We derive the limiting distributions for the test statistics where possible and, for practical needs, we present a permuted samples based approach to find their corresponding non-parametric distributions. Cases where such hypothesis testing presents enough evidence against the null hypothesis of equality of two correlation matrices give rise to the problem of estimating two correlation (or precision) matrices. However, before that we address the statistical problem of estimating conditional dependence between random variables in a single class of samples when data are high-dimensional, which is the second topic of the thesis. We study the graphical lasso method which employs an L1 penalized likelihood expression to estimate the precision matrix and its underlying non-zero graph structure. The lasso penalization termis given by the L1 normof the precisionmatrix elements scaled by a regularization parameter, which determines the trade-off between sparsity of the graph and fit to the data, and its selection is our main focus of investigation. We propose several procedures to select the regularization parameter in the graphical lasso optimization problem that rely on network characteristics such as clustering or connectivity of the graph. Thirdly, we address the more general problem of estimating two precision matrices that are expected to be similar, when datasets are dependent, focusing on the particular case of paired observations. We propose a new method to estimate these precision matrices simultaneously, a weighted fused graphical lasso estimator. The analogous joint estimation method concerning two regression coefficient matrices, which we call weighted fused regression lasso, is also developed in this thesis under the same paired and high-dimensional setting. The two joint estimators maximize penalized marginal log likelihood functions, which encourage both sparsity and similarity in the estimated matrices, and that are solved using an alternating direction method of multipliers (ADMM) algorithm. Sparsity and similarity of thematrices are determined by two tuning parameters and we propose to choose them by controlling the corresponding average error rates related to the expected number of false positive edges in the estimated conditional dependence networks. These testing and estimation methods are implemented within the R package ldstatsHD, and are applied to a comprehensive range of simulated data sets as well as to high-dimensional real case studies of genomic data. We employ testing approaches with the purpose of discovering pathway lists of genes that present significantly different correlation matrices on healthy and unhealthy (e.g., tumor) samples. Besides, we use hypothesis testing problems on correlation sub-matrices to reduce the number of genes for estimation. The proposed joint estimation methods are then considered to find gene interactions that are common between medical conditions as well as interactions that vary in the presence of unhealthy tissues

Essays on Statistical Inference, Nonconvex Optimization and Machine Learning

Over the past decades, numerous optimization and machine learning (ML) algorithms have been proposed, many of which have demonstrated success in real-world applications and significantly impacted people\u27s lives. Researchers have devoted considerable effort to understanding the theoretical underpinnings of these methods and to improving their performance, as well as designing algorithms compatible with demanding real-world constraints. This dissertation focuses on investigating the statistical properties of several mainstream optimization and ML algorithms, enabling us to make decisions with statistical guarantees. First, we examine the classical stochastic gradient descent algorithm (SGD) in a general nonconvex context. Utilizing the multiplier bootstrap technique, we design two inferential procedures that yield consistent covariance matrix estimators and asymptotically exact confidence intervals. Notably, our procedures can be executed online, aligning perfectly with the nature of SGD. We employ fundamentally different proof techniques than those used in inference with convex SGD, and we believe these techniques can be extended to other inferential procedures. Our novel results represent the first practical statistical inference with SGD that transcends the convexity constraint. Second, we explore the problem of testing conditional independence without assuming a specific regression model. In recent years, researchers have proposed numerous model-free statistical testing methods, which are favored for their robustness, particularly in high-dimensional data analysis. Building upon the existing Conditional Randomization Test (CRT), we introduce the Conditional Randomization Rank Test (CRRT). Compared to CRT, CRRT is applicable to a broader range of ML frameworks and offers superior computational efficiency. We demonstrate that CRT can guarantee the desired type 1 error and prove its robustness to distribution misspecification. Through extensive simulations, we empirically validate the effectiveness and robustness of the method. Finally, we investigate a gradient-free extension of the renowned Expectation Maximization algorithm (EM). Although EM and its gradient version have achieved remarkable success in estimating mixture models and other latent variable models, they are not applicable when direct maximization or gradient evaluation is unavailable. To address this limitation, we propose the zeroth-order EM, which requires only function values, making it easily applicable to complex models. We analyze the convergence rate of the zeroth-order EM under both smooth and non-smooth conditions and demonstrate the effectiveness of this method using simulated data