Representation Learning: A Review and New Perspectives

The success of machine learning algorithms generally depends on data representation, and we hypothesize that this is because different representations can entangle and hide more or less the different explanatory factors of variation behind the data. Although specific domain knowledge can be used to help design representations, learning with generic priors can also be used, and the quest for AI is motivating the design of more powerful representation-learning algorithms implementing such priors. This paper reviews recent work in the area of unsupervised feature learning and deep learning, covering advances in probabilistic models, auto-encoders, manifold learning, and deep networks. This motivates longer-term unanswered questions about the appropriate objectives for learning good representations, for computing representations (i.e., inference), and the geometrical connections between representation learning, density estimation and manifold learning

Medical imaging analysis with artificial neural networks

Given that neural networks have been widely reported in the research community of medical imaging, we provide a focused literature survey on recent neural network developments in computer-aided diagnosis, medical image segmentation and edge detection towards visual content analysis, and medical image registration for its pre-processing and post-processing, with the aims of increasing awareness of how neural networks can be applied to these areas and to provide a foundation for further research and practical development. Representative techniques and algorithms are explained in detail to provide inspiring examples illustrating: (i) how a known neural network with fixed structure and training procedure could be applied to resolve a medical imaging problem; (ii) how medical images could be analysed, processed, and characterised by neural networks; and (iii) how neural networks could be expanded further to resolve problems relevant to medical imaging. In the concluding section, a highlight of comparisons among many neural network applications is included to provide a global view on computational intelligence with neural networks in medical imaging

Scaling Up Large-scale Sparse Learning and Its Application to Medical Imaging

abstract: Large-scale $\ell_1$-regularized loss minimization problems arise in high-dimensional applications such as compressed sensing and high-dimensional supervised learning, including classification and regression problems. In many applications, it remains challenging to apply the sparse learning model to large-scale problems that have massive data samples with high-dimensional features. One popular and promising strategy is to scaling up the optimization problem in parallel. Parallel solvers run multiple cores on a shared memory system or a distributed environment to speed up the computation, while the practical usage is limited by the huge dimension in the feature space and synchronization problems. In this dissertation, I carry out the research along the direction with particular focuses on scaling up the optimization of sparse learning for supervised and unsupervised learning problems. For the supervised learning, I firstly propose an asynchronous parallel solver to optimize the large-scale sparse learning model in a multithreading environment. Moreover, I propose a distributed framework to conduct the learning process when the dataset is distributed stored among different machines. Then the proposed model is further extended to the studies of risk genetic factors for Alzheimer's Disease (AD) among different research institutions, integrating a group feature selection framework to rank the top risk SNPs for AD. For the unsupervised learning problem, I propose a highly efficient solver, termed Stochastic Coordinate Coding (SCC), scaling up the optimization of dictionary learning and sparse coding problems. The common issue for the medical imaging research is that the longitudinal features of patients among different time points are beneficial to study together. To further improve the dictionary learning model, I propose a multi-task dictionary learning method, learning the different task simultaneously and utilizing shared and individual dictionary to encode both consistent and changing imaging features.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Altered effective connectivity in sensorimotor cortices is a signature of severity and clinical course in depression

Published September 30, 2021.Functional neuroimaging research on depression has traditionally targeted neural networks associated with the psychological aspects of depression. In this study, instead, we focus on alterations of sensorimotor function in depression. We used restingstate functional MRI data and dynamic causal modeling (DCM) to assess the hypothesis that depression is associated with aberrant effective connectivity within and between key regions in the sensorimotor hierarchy. Using hierarchical modeling of betweensubject effects in DCM with parametric empirical Bayes we first established the architecture of effective connectivity in sensorimotor cortices.We found that in (interoceptive and exteroceptive) sensory cortices across participants, the backward connections are predominantly inhibitory, whereas the forward connections are mainly excitatory in nature. In motor cortices these parities were reversed. With increasing depression severity, these patterns are depreciated in exteroceptive and motor cortices and augmented in the interoceptive cortex, an observation that speaks to depressive symptomatology. We established the robustness of these results in a leave-one-out cross-validation analysis and by reproducing the main results in a follow-up dataset. Interestingly, with (nonpharmacological) treatment, depression-associated changes in backward and forward effective connectivity partially reverted to group mean levels. Overall, altered effective connectivity in sensorimotor cortices emerges as a promising and quantifiable candidate marker of depression severity and treatment response.This research was supported by the Basque Government through the Basque Excellence Research Centres 2018-2021 program; the Spanish Ministry of Science, Innovation, and Universities (Basque Center on Cognition, Brain and Language Severo Ochoa excellence accreditation SEV-2015-0490 and Basque Center of Applied Mathematics (BCAM) Severo Ochoa accreditation SEV-2017-0718); and project MTM2017-82379-R (Agencia Estatal de Investigación/Fondo Europeo de Desarrollo Regional, Unión Europea; principal investigator Dr. Maria Xose Rodriguez, BCAM). Data collection was funded by Russian Science Foundation Grant 16-15-00 183. K.J.F. was funded by a Wellcome Trust Principal Research Fellowship (Reference 088130/Z/09/Z

Altered effective connectivity in sensorimotor cortices is a signature of severity and clinical course in depression

Functional neuroimaging research on depression has traditionally targeted neural networks associated with the psychological aspects of depression. In this study, instead, we focus on alterations of sensorimotor function in depression. We used resting-state functional MRI data and dynamic causal modeling (DCM) to assess the hypothesis that depression is associated with aberrant effective connectivity within and between key regions in the sensorimotor hierarchy. Using hierarchical modeling of between-subject effects in DCM with parametric empirical Bayes we first established the architecture of effective connectivity in sensorimotor cortices. We found that in (interoceptive and exteroceptive) sensory cortices across participants, the backward connections are predominantly inhibitory, whereas the forward connections are mainly excitatory in nature. In motor cortices these parities were reversed. With increasing depression severity, these patterns are depreciated in exteroceptive and motor cortices and augmented in the interoceptive cortex, an observation that speaks to depressive symptomatology. We established the robustness of these results in a leave-one-out cross-validation analysis and by reproducing the main results in a follow-up dataset. Interestingly, with (nonpharmacological) treatment, depression-associated changes in backward and forward effective connectivity partially reverted to group mean levels. Overall, altered effective connectivity in sensorimotor cortices emerges as a promising and quantifiable candidate marker of depression severity and treatment response

Bilinearity, Rules, and Prefrontal Cortex

Humans can be instructed verbally to perform computationally complex cognitive tasks; their performance then improves relatively slowly over the course of practice. Many skills underlie these abilities; in this paper, we focus on the particular question of a uniform architecture for the instantiation of habitual performance and the storage, recall, and execution of simple rules. Our account builds on models of gated working memory, and involves a bilinear architecture for representing conditional input-output maps and for matching rules to the state of the input and working memory. We demonstrate the performance of our model on two paradigmatic tasks used to investigate prefrontal and basal ganglia function

A stochastic algorithm for probabilistic independent component analysis

The decomposition of a sample of images on a relevant subspace is a recurrent problem in many different fields from Computer Vision to medical image analysis. We propose in this paper a new learning principle and implementation of the generative decomposition model generally known as noisy ICA (for independent component analysis) based on the SAEM algorithm, which is a versatile stochastic approximation of the standard EM algorithm. We demonstrate the applicability of the method on a large range of decomposition models and illustrate the developments with experimental results on various data sets.Comment: Published in at http://dx.doi.org/10.1214/11-AOAS499 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org

Recent Advances in Signal Processing

The signal processing task is a very critical issue in the majority of new technological inventions and challenges in a variety of applications in both science and engineering fields. Classical signal processing techniques have largely worked with mathematical models that are linear, local, stationary, and Gaussian. They have always favored closed-form tractability over real-world accuracy. These constraints were imposed by the lack of powerful computing tools. During the last few decades, signal processing theories, developments, and applications have matured rapidly and now include tools from many areas of mathematics, computer science, physics, and engineering. This book is targeted primarily toward both students and researchers who want to be exposed to a wide variety of signal processing techniques and algorithms. It includes 27 chapters that can be categorized into five different areas depending on the application at hand. These five categories are ordered to address image processing, speech processing, communication systems, time-series analysis, and educational packages respectively. The book has the advantage of providing a collection of applications that are completely independent and self-contained; thus, the interested reader can choose any chapter and skip to another without losing continuity