A Survey on Deep Learning in Medical Image Analysis

Deep learning algorithms, in particular convolutional networks, have rapidly become a methodology of choice for analyzing medical images. This paper reviews the major deep learning concepts pertinent to medical image analysis and summarizes over 300 contributions to the field, most of which appeared in the last year. We survey the use of deep learning for image classification, object detection, segmentation, registration, and other tasks and provide concise overviews of studies per application area. Open challenges and directions for future research are discussed.Comment: Revised survey includes expanded discussion section and reworked introductory section on common deep architectures. Added missed papers from before Feb 1st 201

MACHINE LEARNING BASED ANALYSIS AND COMPUTER AIDED CLASSIFICATION OF NEUROPSYCHIATRIC DISORDERS USING NEUROIMAGING

Machine learning (ML) based analysis of neuroimages in neuropsychiatry context are advancing the understanding of neurobiological profiles and the pathological bases of neuropsychiatric disorders. Computational analysis and investigations on features derived from structural magnetic resonance imaging (sMRI) of the brain are used to quantify morphological or anatomical characteristics of the different regions of the brain that have role in several distinct brain functions. This helps in the realization of anatomical underpinnings of those disorders that cause brain atrophy. Structural neuroimaging data acquired from schizophrenia (SCZ), bipolar disorder (BD) patients and people who experienced psychosis for the first time, are used for the experiments presented in this thesis. The cerebral cortex (i.e., gray matter) of the brain is one of the most studied anatomical part using 'cortical-average-thickness' distribution feature in the literature. This helps in the realization of the anatomical underpinning of those mental illnesses that cause brain atrophy. To this regard, based on statistical background, 'cortical-skewness' feature, a novel digital imaging-derived neuroanatomical biomarker that could potentially assist in the differentiation of healthy control (HC) and patient groups is proposed and tested in this thesis. The core theme of machine intelligence relies in extracting and learning patterns of input data from experience. Classification is one of the task. In a basic set up, ML algorithms are trained using exemplary multivariate data features and its associated class labels, so that they could be able to create models and do predictive classification and other tasks. Considering the conundrum nature of psychiatric disorders, researchers in the field, could benefit from ML based analysis of complex brain patterns. Out of many, one task is computer aided classification (CAC). This is achieved by training the algorithms, these complex brain patterns and their corresponding diagnostic statistics manual (DSM) based clinical gold standard labels. Indeed, in the literature, supervised learning methods such as support vector machines (SVM) which follow inductive learning strategy are widely exploited and achieved interesting results. Observing this and due to the fact that the most widely available relevant anatomical features of the cortex such as thickness and volume values, could not be considered satisfactory features because of the heterogeneous nature of the human brain anatomy due to differences in age, gender etc., a contextual similarity based learning is proposed. This learning uses a transductive learning mechanism (i.e, learn a specific function for the problem at hand) instead of learning a general function to solve a specific problem. Based on this, it is adopted, a formulation of a semi supervised graph transduction (label propagation) algorithm based on the notions of game theory, where the consistent labeling is represented with Nash equilibrium, to tackle the problem of learning from neuroimages with subtle microscopic difference among different clinical groups. However, since such kind of algorithms heavily rely on the graph structure of the extracted features, we extended the classification procedure by introducing a pre-training phase based on a distance metric learning strategy with the aim of enhancing the contextual similarity of the images by providing a 'must belong in the same class' and 'must not belong in the same class' constraint from the available training data. This would result to increase intra-class similarity and decrease inter-class similarity. The proposed classification pipeline is used for searching anatomical biomarkers. With the goal of identifying potential neuroanatomical markers of a psychiatric disorder, it is aimed to develop a feature selection strategy taking into consideration the widely exploited cortical thickness and the proposed skewness feature, with the objective of searching a combination of features from all cortical regions of the brain that could maximize the possible differentiation among the different clinical groups Considering Research Domain Criteria (RDoC) framework developed by National Institute of Mental Health (NIMH) with the aim of developing biologically valid perspective of mental disorders by integrating multimodal sources, clinical interview scores and neuroimaging data are used with ML methods to tackle the challenging problem of differential classification of BD vs. SCZ. Finally, as deep learning methods are emerging with remarkable results in several application domains, we adopted this class of methods especially convolutional neural networks (CNNs) with a 3D approach, to extract volumetric neuroanatomical markers. CAC of first episode psychosis (FEP) is performed by exploiting the 3D complex spatial structure of the brain to identify key regions of the brain associated with the pathophysiology of FEP. Testing of individualized predictions with big dataset of 855 structural scans to identify possible markers of the disease is performed

Change blindness: eradication of gestalt strategies

Arrays of eight, texture-defined rectangles were used as stimuli in a one-shot change blindness (CB) task where there was a 50% chance that one rectangle would change orientation between two successive presentations separated by an interval. CB was eliminated by cueing the target rectangle in the first stimulus, reduced by cueing in the interval and unaffected by cueing in the second presentation. This supports the idea that a representation was formed that persisted through the interval before being 'overwritten' by the second presentation (Landman et al, 2003 Vision Research 43149–164]. Another possibility is that participants used some kind of grouping or Gestalt strategy. To test this we changed the spatial position of the rectangles in the second presentation by shifting them along imaginary spokes (by ±1 degree) emanating from the central fixation point. There was no significant difference seen in performance between this and the standard task [F(1,4)=2.565, p=0.185]. This may suggest two things: (i) Gestalt grouping is not used as a strategy in these tasks, and (ii) it gives further weight to the argument that objects may be stored and retrieved from a pre-attentional store during this task

Glucose enhancement of human memory: A comprehensive research review of the glucose memory facilitation effect

The brain relies upon glucose as its primary fuel. In recent years, a rich literature has developed from both human and animal studies indicating that increases in circulating blood glucose can facilitate cognitive functioning. This phenomenon has been termed the ‘glucose memory facilitation effect’. The purpose of this review is to discuss a number of salient studies which have investigated the influence of glucose ingestion on neurocognitive performance in individuals with (a) compromised neurocognitive capacity, as well as (b) normally functioning individuals (with a focus on research conducted with human participants). The proposed neurocognitive mechanisms purported to underlie the modulatory effect of glucose on neurocognitive performance will also be considered. Many theories have focussed upon the hippocampus, given that this brain region is heavily implicated in learning and memory. Further, it will be suggested that glucose is a possible mechanism underlying the phenomenon that enhanced memory performance is typically observed for emotionally laden stimuli

Basic prediction mechanisms as a precursor for schizophrenia studies

Traditionally, early visual cortex (V1-3) was thought of as merely a relay centre for feedforward retinal input, providing entry to the cortical visual processing steam. However, in addition to feedforward retinal input, V1 receives a large amount of intracortical information through feedback and lateral connections. Human visual perception is constructed from combining feedforward inputs with these feedback and lateral contributions. Feedback connections allow the visual cortical response to feedforward information to be affected by expectation, knowledge, and context; even at the level of early visual cortex. In Chapter 1 we discuss the feedforward and feedback visual processing streams. We consider historical philosophical and scientific propositions about constructive vision. We introduce modern theories of constructive vision, which suggest that vision is an active process that aims to infer or predict the cause of sensory inputs. We discuss how V1 therefore represents not only retinal input but also high-level effects related to constructive predictive perception. Visual illusions are a ‘side effect’ of constructive and inferential visual perception. For the vast majority of stimulus inputs, integration with context and knowledge facilitates clearer, more veridical perception. In illusion these constructive mechanisms produce incorrect percepts. Illusory effects can be observed in early visual cortex, even when there is no change in the feedforward visual input. We suggest that illusions therefore provide us with a tool to probe feedforward and feedback integration, as they exploit the difference between retinal stimulation and resulting perception. Thus, illusions allow us to see the changes in activation and perception induced only by feedback without changes in feedforward input. We discuss a few specific examples of illusion generation through feedback and the accompanying effects on V1 processing. In Schizophrenia, the integration of feedback and feedforward information is thought to be dysfunctional, with unbalanced contributions of the two sources. This is evidenced by disrupted contextual binding in visual perception and corresponding deficits in contextual illusion perception. We propose that illusions can provide a window into constructive and inferential visual perception in Schizophrenia. Use of illusion paradigms could help elucidate the deficits existing within feedback and feedforward integration. If we can establish clear effects of illusory feedback to V1 in a typical population, we can apply this knowledge to clinical subjects to observe the differences in feedback and feedforward information. Chapter 2 describes a behavioural study of the rubber hand illusion. We probe how multimodal illusory experience arises under varying reliabilities of visuotactile feedforward input. We recorded Likert ratings of illusion experience from subjects, after their hidden hand was stimulated either synchronously or asynchronously with a visible rubber hand (200, 300, 400, or 600ms visuotactile asynchronicity). We used two groups, assessed by a questionnaire measuring a subject’s risk of developing Schizophrenia - moderate/high scorers and a control group of zero-scorers. We therefore consider how schizotypal symptoms contribute to rubber hand illusory experience and interact with visuotactile reliability. Our results reveal that the impact of feedforward information on higher level illusory body schema is modulated by its reliability. Less reliable feedforward inputs (increasing asynchronicity) reduce illusion perception. Our data suggests that some illusions may not be affected on a spectrum of schizotypal traits but only in the full schizophrenic disorder, as we found no effect of group on illusion perception. In Chapter 3 we present an fMRI investigation of the rubber hand illusion in typical participants. Cortical feedback allows information about other modalities and about cognitive states to be represented at the level of V1. Using a multimodal illusion, we investigated whether crossmodal and illusory states could be represented in early visual cortex in the absence of differential visual input. We found increased BOLD activity in motion area V5 and global V1 when the feedforward tactile information and the illusory outcome were incoherent (for example when the subject was experiencing the illusion during asynchronous stimulation). This is suggestive of increased predictive error, supporting predictive coding models of cognitive function. Additionally, we reveal that early visual cortex contains pattern representations specific to the illusory state, irrespective of tactile stimulation and under identical feedforward visual input. In Chapter 4 we use the motion-induced blindness illusion to demonstrate that feedback modulates stimulus representations in V1 during illusory disappearance. We recorded fMRI data from subjects viewing a 2D cross array rotating around a central axis, passing over an oriented Gabor patch target (45°/ 135°). We attempted to decode the target orientation from V1 when the target was either visible or invisible to subjects. Target information could be decoded during target visibility but not during motion-induced blindness. This demonstrates that the target representation in V1 is distorted or destroyed when the target is perceptually invisible. This illusion therefore has effects not only at higher cortical levels, as previously shown, but also in early sensory areas. The representation of the stimulus in V1 is related to perceptual awareness. Importantly, Chapter 4 demonstrated that intracortical processing can disturb constant feedforward information and overwrite feedforward representations. We suggest that the distortion observed occurs through feedback from V5 about the cross array in motion, overwriting feedforward orientation information. The flashed face distortion illusion is a relatively newly discovered illusion in which quickly presented faces become monstrously distorted. The neural underpinnings of the illusion remain unclear; however it has been hypothesised to be a face-specific effect. In Chapter 5 we challenged this account by exploiting two hallmarks of face-specific processing - the other-race effect and left visual field superiority. In two experiments, two ethnic groups of subjects viewed faces presented bilaterally in the visual periphery. We varied the race of the faces presented (same or different than subject), the visual field that the faces were presented in, and the duration of successive presentations (250, 500, 750 or 1000ms per face before replacement). We found that perceived distortion was not affected by stimulus race, visual field, or duration of successive presentations (measured by forced choice in experiment 1 and Likert scale in experiment 2). We therefore provide convincing evidence that FFD is not face-specific and instead suggest that it is an object-general effect created by comparisons between successive stimuli. These comparisons are underlined by a fed back higher level model which dictates that objects cannot immediately replace one another in the same retinotopic space without movement. In Chapter 6 we unify these findings. We discuss how our data show fed back effects on perception to produce visual illusion; effects which cannot be explained through purely feedforward activity processing. We deliberate how lateral connections and attention effects may contribute to our results. We describe known neural mechanisms which allow for the integration of feedback and feedforward information. We discuss how this integration allows V1 to represent the content of visual awareness, including during some of the illusions presented in this thesis. We suggest that a unifying theory of brain computation, Predictive Coding, may explain why feedback exerts top-down effects on feedforward processing. Lastly we discuss how our findings, and others that demonstrate feedback and prediction effects, could help develop the study and understanding of schizophrenia, including our understanding of the underlying neurological pathologies

The Role of Chromatin Plasticity in Schizophrenia and Anxiety Diseases

Schizophrenia is a severe neuropsychiatric disorder with high phenotypic complexity and multifactorial inheritance. Cognitive dysfunctions have been identified as the core feature of the disease and they are resistant to treatment with available antipsychotics. Impaired working memory and disrupted sensorimotor gating are the cognitive hallmarks of schizophrenia. Both of these cognitive dysfunctions are defined as cognitive endophenotypes that present a biomarker and guidepost for identification of the cause and course of schizophrenia. The etiopathogenesis of schizophrenia is thought to rely on genome and environment (GxE) interactions. Epigenetic enzymes such as histone-deacetylases (HDACs) are key mediators of GxE interactions. HDACs remove acetyl-groups of histone-proteins in response to environment stimuli, thereby changing the chromatin structure resulting into differential gene-expression important for cognition. Deregulated histone-acetylation leads to impairments in learning and memory. Two independent human post-mortem studies have reported elevated HDAC1 levels in the hippocampus and prefrontal cortex of individuals with schizophrenia, with both brain regions being important for the regulation of cognitive endophenotypes of schizophrenia. My results showed, that overexpression of neuronal HDAC1 in the prefrontal cortex of adult mice resulted in schizophrenia-like symptoms such as increased anxiety, depressive-like behavior, impaired fear extinction and cognitive endophenotypes such as impaired working memory performance and deficits in sensorimotor gating function. Inhibition of HDAC1 ameliorated such phenotypes. Moreover, environmental risk factors for schizophrenia such as early life stress induced cognitive endophenotypes of schizophrenia and mediated the up-regulation of prefrontal cortical HDAC1, simulating the situation observed in the post-mortem prefrontal cortex tissue of individuals with schizophrenia. . Interestingly, while manipulating neuronal HDAC1 levels in the prefrontal cortex of mice caused schizophrenia-like phenotypes, affecting neuronal HDAC1 levels in the dorsal hippocampus had no impact on such behaviors. Instead under physiological conditions, HDAC1 in the dorsal hippocampus regulates the extinction of fear memories in mice by transcriptional repression of Immediate Early Genes (IEG´s). In conclusion, these data indicate a brain-region specific function of HDAC1 in cognition and emotional behavior and provide important knowledge on the role of HDAC1 in the adult brain

Deep Learning in Medical Image Analysis

The computer-assisted analysis for better interpreting images have been longstanding issues in the medical imaging field. On the image-understanding front, recent advances in machine learning, especially, in the way of deep learning, have made a big leap to help identify, classify, and quantify patterns in medical images. Specifically, exploiting hierarchical feature representations learned solely from data, instead of handcrafted features mostly designed based on domain-specific knowledge, lies at the core of the advances. In that way, deep learning is rapidly proving to be the state-of-the-art foundation, achieving enhanced performances in various medical applications. In this article, we introduce the fundamentals of deep learning methods; review their successes to image registration, anatomical/cell structures detection, tissue segmentation, computer-aided disease diagnosis or prognosis, and so on. We conclude by raising research issues and suggesting future directions for further improvements