Modular Networks: Learning to Decompose Neural Computation

Scaling model capacity has been vital in the success of deep learning. For a typical network, necessary compute resources and training time grow dramatically with model size. Conditional computation is a promising way to increase the number of parameters with a relatively small increase in resources. We propose a training algorithm that flexibly chooses neural modules based on the data to be processed. Both the decomposition and modules are learned end-to-end. In contrast to existing approaches, training does not rely on regularization to enforce diversity in module use. We apply modular networks both to image recognition and language modeling tasks, where we achieve superior performance compared to several baselines. Introspection reveals that modules specialize in interpretable contexts.Comment: NIPS 201

Modular Networks: Learning to Decompose Neural Computation

Scaling model capacity has been vital in the success of deep learning. For a typical network, necessary compute resources and training time grow dramatically with model size. Conditional computation is a promising way to increase the number of parameters with a relatively small increase in resources. We propose a training algorithm that flexibly chooses neural modules based on the data to be processed. Both the decomposition and modules are learned end-to-end. In contrast to existing approaches, training does not rely on regularization to enforce diversity in module use. We apply modular networks both to image recognition and language modeling tasks, where we achieve superior performance compared to several baselines. Introspection reveals that modules specialize in interpretable contexts

Joint sparse representation of brain activity patterns in multi-task fMRI data

A single-task functional magnetic resonance imaging (fMRI) experiment may only partially highlight alterations to functional brain networks affected by a particular disorder. Multivariate analysis across multiple fMRI tasks may increase the sensitivity of fMRI-based diagnosis. Prior research using multi-task analysis in fMRI, such as those that use joint independent component analysis (jICA), has mainly assumed that brain activity patterns evoked by different tasks are independent. This may not be valid in practice. Here, we use sparsity, which is a natural characteristic of fMRI data in the spatial domain, and propose a joint sparse representation analysis (jSRA) method to identify common information across different functional subtraction (contrast) images in data from a multi-task fMRI experiment. Sparse representation methods do not require independence, or that the brain activity patterns be nonoverlapping. We use functional subtraction images within the joint sparse representation analysis to generate joint activation sources and their corresponding sparse modulation profiles. We evaluate the use of sparse representation analysis to capture individual differences with simulated fMRI data and with experimental fMRI data. The experimental fMRI data was acquired from 16 young (age: 19-26) and 16 older (age: 57-73) adults obtained from multiple speech comprehension tasks within subjects, where an independent measure (namely, age in years) can be used to differentiate between groups. Simulation results show that this method yields greater sensitivity, precision, and higher Jaccard indexes (which measures similarity and diversity of the true and estimated brain activation sources) than does the jICA method. Moreover, superiority of the jSRA method in capturing individual differences was successfully demonstrated using experimental fMRI data

New Statistical Methods for Evaluating Brain Functional Connectivity

The human brain functions through the coordination of a complex network of billions of neurons. This network, when defined by the functions it dictates, is known as functional brain connectivity. Associating brain networks with clinical symptoms and outcomes has great potential for shaping future work in neuroimaging and clinical practice. Resting-state functional magnetic resonance imaging (rfMRI) has commonly been used to establish the functional brain network; however, understanding the links to clinical characteristics is still an ongoing research question. Existing methods for analysis of functional brain networks, such as independent component analysis and canonical correlation analysis, have laid a good foundation for this research; yet most methods do not directly model the node-level association between connectivity and clinical characteristics, and thus provide limited ability for interpretation. To address those limitations, this dissertation research focuses on developing efficient methods that identify node-level associations to answer important research questions in brain imaging studies. In the first project, we propose a joint modeling framework for estimating functional connectivity networks from rfMRI time series data and evaluating the predictability of individual's brain connectivity patterns using their clinical characteristics. Our goal is to understand the link between clinical presentations of psychiatric disorders and functional brain connectivity at different region pairs. Our modeling framework consists of two components: estimation of individual functional connectivity networks and identifying associations with clinical characteristics. We propose a model fitting procedure for jointly estimating these components via the alternating direction method of multipliers (ADMM) algorithm. The key advantage of the proposed approach lies in its ability to directly identify the brain region pairs between which the functional connectivity is strongly associated with the clinical characteristics. Compared to existing methods, our framework has the flexibility to integrate machine learning methods to estimate the nonlinear predictive effects of clinical characteristics. Additionally, jointly modeling the precision matrix and the predictive model estimates provides a novel framework to accommodate the uncertainty in estimating functional connectivity. In the second project, we focus on a scalar-on-network regression problem which utilizes brain functional connectivity networks to predict a single clinical outcome of interest, where the regression coefficient is edge-dependent. To improve estimation efficiency, we develop a two stage boosting algorithm to estimate the sparse edge-dependent regression coefficients by leveraging the knowledge of brain functional organization. Simulations have shown the proposed method has higher power to detect the true signals while controlling the false discovery rate better than existing approaches. We apply the proposed method to analysis of rfMRI data in the Adolescent Brain Cognitive Development (ABCD) study and identify the important functional connectivity sub-networks that are associated with general cognitive ability. In the third project, we extend scalar-on-network regression via boosting in the second project by relaxing the homogeneity constraints within the prespecified functional connectivity networks. We adopt deep neural networks (DNN) to model the edge-dependent regression coefficients in light of the edge-level and node level features in the brain network, as well as the well-known brain functional organization. In addition, the proposed DNN-based scalar-on-network regression has the flexibility to incorporate the signal pattern from other imaging modalities into the model. We develop an efficient model fitting method based on ADMM. The proposed method is evaluated and compared with existing alternatives via simulations and analysis of rfMRI and task fMRI data in the ABCD study.PHDBiostatisticsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/169684/1/emorrisl_1.pd

Sparse and low-rank techniques for the efficient restoration of images

Image reconstruction is a key problem in numerous applications of computer vision and medical imaging. By removing noise and artifacts from corrupted images, or by enhancing the quality of low-resolution images, reconstruction methods are essential to provide high-quality images for these applications. Over the years, extensive research efforts have been invested toward the development of accurate and efficient approaches for this problem. Recently, considerable improvements have been achieved by exploiting the principles of sparse representation and nonlocal self-similarity. However, techniques based on these principles often suffer from important limitations that impede their use in high-quality and large-scale applications. Thus, sparse representation approaches consider local patches during reconstruction, but ignore the global structure of the image. Likewise, because they average over groups of similar patches, nonlocal self-similarity methods tend to over-smooth images. Such methods can also be computationally expensive, requiring a hour or more to reconstruct a single image. Furthermore, existing reconstruction approaches consider either local patch-based regularization or global structure regularization, due to the complexity of combining both regularization strategies in a single model. Yet, such combined model could improve upon existing techniques by removing noise or reconstruction artifacts, while preserving both local details and global structure in the image. Similarly, current approaches rarely consider external information during the reconstruction process. When the structure to reconstruct is known, external information like statistical atlases or geometrical priors could also improve performance by guiding the reconstruction. This thesis addresses limitations of the prior art through three distinct contributions. The first contribution investigates the histogram of image gradients as a powerful prior for image reconstruction. Due to the trade-off between noise removal and smoothing, image reconstruction techniques based on global or local regularization often over-smooth the image, leading to the loss of edges and textures. To alleviate this problem, we propose a novel prior for preserving the distribution of image gradients modeled as a histogram. This prior is combined with low-rank patch regularization in a single efficient model, which is then shown to improve reconstruction accuracy for the problems of denoising and deblurring. The second contribution explores the joint modeling of local and global structure regularization for image restoration. Toward this goal, groups of similar patches are reconstructed simultaneously using an adaptive regularization technique based on the weighted nuclear norm. An innovative strategy, which decomposes the image into a smooth component and a sparse residual, is proposed to preserve global image structure. This strategy is shown to better exploit the property of structure sparsity than standard techniques like total variation. The proposed model is evaluated on the problems of completion and super-resolution, outperforming state-of-the-art approaches for these tasks. Lastly, the third contribution of this thesis proposes an atlas-based prior for the efficient reconstruction of MR data. Although popular, image priors based on total variation and nonlocal patch similarity often over-smooth edges and textures in the image due to the uniform regularization of gradients. Unlike natural images, the spatial characteristics of medical images are often restricted by the target anatomical structure and imaging modality. Based on this principle, we propose a novel MRI reconstruction method that leverages external information in the form of an probabilistic atlas. This atlas controls the level of gradient regularization at each image location, via a weighted total-variation prior. The proposed method also exploits the redundancy of nonlocal similar patches through a sparse representation model. Experiments on a large scale dataset of T1-weighted images show this method to be highly competitive with the state-of-the-art