Novel Image Representations and Learning Tasks

abstract: Computer Vision as a eld has gone through signicant changes in the last decade. The eld has seen tremendous success in designing learning systems with hand-crafted features and in using representation learning to extract better features. In this dissertation some novel approaches to representation learning and task learning are studied. Multiple-instance learning which is generalization of supervised learning, is one example of task learning that is discussed. In particular, a novel non-parametric k- NN-based multiple-instance learning is proposed, which is shown to outperform other existing approaches. This solution is applied to a diabetic retinopathy pathology detection problem eectively. In cases of representation learning, generality of neural features are investigated rst. This investigation leads to some critical understanding and results in feature generality among datasets. The possibility of learning from a mentor network instead of from labels is then investigated. Distillation of dark knowledge is used to eciently mentor a small network from a pre-trained large mentor network. These studies help in understanding representation learning with smaller and compressed networks.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Restauration d'images en IRM anatomique pour l'étude préclinique des marqueurs du vieillissement cérébral

Les maladies neurovasculaires et neurodégénératives liées à l'âge sont en forte augmentation. Alors que ces changements pathologiques montrent des effets sur le cerveau avant l'apparition de symptômes cliniques, une meilleure compréhension du processus de vieillissement normal du cerveau aidera à distinguer l'impact des pathologies connues sur la structure régionale du cerveau. En outre, la connaissance des schémas de rétrécissement du cerveau dans le vieillissement normal pourrait conduire à une meilleure compréhension de ses causes et peut-être à des interventions réduisant la perte de fonctions cérébrales associée à l'atrophie cérébrale. Par conséquent, ce projet de thèse vise à détecter les biomarqueurs du vieillissement normal et pathologique du cerveau dans un modèle de primate non humain, le singe marmouset (Callithrix Jacchus), qui possède des caractéristiques anatomiques plus proches de celles des humains que de celles des rongeurs. Cependant, les changements structurels (par exemple, de volumes, d'épaisseur corticale) qui peuvent se produire au cours de leur vie adulte peuvent être minimes à l'échelle de l'observation. Dans ce contexte, il est essentiel de disposer de techniques d'observation offrant un contraste et une résolution spatiale suffisamment élevés et permettant des évaluations détaillées des changements morphométriques du cerveau associé au vieillissement. Cependant, l'imagerie de petits cerveaux dans une plateforme IRM 3T dédiée à l'homme est une tâche difficile car la résolution spatiale et le contraste obtenus sont insuffisants par rapport à la taille des structures anatomiques observées et à l'échelle des modifications attendues. Cette thèse vise à développer des méthodes de restauration d'image pour les images IRM précliniques qui amélioreront la robustesse des algorithmes de segmentation. L'amélioration de la résolution spatiale des images à un rapport signal/bruit constant limitera les effets de volume partiel dans les voxels situés à la frontière entre deux structures et permettra une meilleure segmentation tout en augmentant la reproductibilité des résultats. Cette étape d'imagerie computationnelle est cruciale pour une analyse morphométrique longitudinale fiable basée sur les voxels et l'identification de marqueurs anatomiques du vieillissement cérébral en suivant les changements de volume dans la matière grise, la matière blanche et le liquide cérébral.Age-related neurovascular and neurodegenerative diseases are increasing significantly. While such pathological changes show effects on the brain before clinical symptoms appear, a better understanding of the normal aging brain process will help distinguish known pathologies' impact on regional brain structure. Furthermore, knowledge of the patterns of brain shrinkage in normal aging could lead to a better understanding of its causes and perhaps to interventions reducing the loss of brain functions. Therefore, this thesis project aims to detect normal and pathological brain aging biomarkers in a non-human primate model, the marmoset monkey (Callithrix Jacchus) which possesses anatomical characteristics more similar to humans than rodents. However, structural changes (e.g., volumes, cortical thickness) that may occur during their adult life may be minimal with respect to the scale of observation. In this context, it is essential to have observation techniques that offer sufficiently high contrast and spatial resolution and allow detailed assessments of the morphometric brain changes associated with aging. However, imaging small brains in a 3T MRI platform dedicated to humans is a challenging task because the spatial resolution and the contrast obtained are insufficient compared to the size of the anatomical structures observed and the scale of the xpected changes with age. This thesis aims to develop image restoration methods for preclinical MR images that will improve the robustness of the segmentation algorithms. Improving the resolution of the images at a constant signal-to-noise ratio will limit the effects of partial volume in voxels located at the border between two structures and allow a better segmentation while increasing the results' reproducibility. This computational imaging step is crucial for a reliable longitudinal voxel-based morphometric analysis and for the identification of anatomical markers of brain aging by following the volume changes in gray matter, white matter and cerebrospinal fluid

High-speed imaging in fluids

High-speed imaging is in popular demand for a broad range of experiments in fluids. It allows for a detailed visualization of the event under study by acquiring a series of image frames captured at high temporal and spatial resolution. This review covers high-speed imaging basics, by defining criteria for high-speed imaging experiments in fluids and to give rule-of-thumbs for a series of cases. It also considers stroboscopic imaging, triggering and illumination, and scaling issues. It provides guidelines for testing and calibration. Ultra high-speed imaging at frame rates exceeding 1 million frames per second is reviewed, and the combination of conventional experiments in fluids techniques with high-speed imaging techniques are discussed. The review is concluded with a high-speed imaging chart, which summarizes criteria for temporal scale and spatial scale and which facilitates the selection of a high-speed imaging system for the applicatio

Informatics for EEG biomarker discovery in clinical neuroscience

Neurological and developmental disorders (NDDs) impose an enormous burden of disease on children throughout the world. Two of the most common are autism spectrum disorder (ASD) and epilepsy. ASD has recently been estimated to affect 1 in 68 children, making it the most common neurodevelopmental disorder in children. Epilepsy is also a spectrum disorder that follows a developmental trajectory, with an estimated prevalence of 1%, nearly as common as autism. ASD and epilepsy co-occur in approximately 30% of individuals with a primary diagnosis of either disorder. Although considered to be different disorders, the relatively high comorbidity suggests the possibility of common neuropathological mechanisms. Early interventions for NDDs lead to better long-term outcomes. But early intervention is predicated on early detection. Behavioral measures have thus far proven ineffective in detecting autism before about 18 months of age, in part because the behavioral repertoire of infants is so limited. Similarly, no methods for detecting emerging epilepsy before seizures begin are currently known. Because atypical brain development is likely to precede overt behavioral manifestations by months or even years, a critical developmental window for early intervention may be opened by the discovery of brain based biomarkers. Analysis of brain activity with EEG may be under-utilized for clinical applications, especially for neurodevelopment. The hypothesis investigated in this dissertation is that new methods of nonlinear signal analysis, together with methods from biomedical informatics, can extract information from EEG data that enables detection of atypical neurodevelopment. This is tested using data collected at Boston Children’s Hospital. Several results are presented. First, infants with a family history of ASD were found to have EEG features that may enable autism to be detected as early as 9 months. Second, significant EEG-based differences were found between children with absence epilepsy, ASD and control groups using short 30-second EEG segments. Comparison of control groups using different EEG equipment supported the claim that EEG features could be computed that were independent of equipment and lab conditions. Finally, the potential for this technology to help meet the clinical need for neurodevelopmental screening and monitoring in low-income regions of the world is discussed

Modularity in artificial neural networks

Artificial neural networks are deep machine learning models that excel at complex artificial intelligence tasks by abstracting concepts through multiple layers of feature extraction. Modular neural networks are artificial neural networks that are composed of multiple subnetworks called modules. The study of modularity has a long history in the field of artificial neural networks and many of the actively studied models in the domain of artificial neural networks have modular aspects. In this work, we aim to formalize the study of modularity in artificial neural networks and outline how modularity can be used to enhance some neural network performance measures. We do an extensive review of the current practices of modularity in the literature. Based on that, we build a framework that captures the essential properties characterizing the modularization process. Using this modularization framework as an anchor, we investigate the use of modularity to solve three different problems in artificial neural networks: balancing latency and accuracy, reducing model complexity and increasing robustness to noise and adversarial attacks. Artificial neural networks are high-capacity models with high data and computational demands. This represents a serious problem for using these models in environments with limited computational resources. Using a differential architectural search technique, we guide the modularization of a fully-connected network into a modular multi-path network. By evaluating sampled architectures, we can establish a relation between latency and accuracy that can be used to meet a required soft balance between these conflicting measures. A related problem is reducing the complexity of neural network models while minimizing accuracy loss. CapsNet is a neural network architecture that builds on the ideas of convolutional neural networks. However, the original architecture is shallow and has wide layers that contribute significantly to its complexity. By replacing the early wide layers by parallel deep independent paths, we can significantly reduce the complexity of the model. Combining this modular architecture with max-pooling, DropCircuit regularization and a modified variant of the routing algorithm, we can achieve lower model latency with the same or better accuracy compared to the baseline. The last problem we address is the sensitivity of neural network models to random noise and to adversarial attacks, a highly disruptive form of engineered noise. Convolutional layers are the basis of state-of-the-art computer vision models and, much like other neural network layers, they suffer from sensitivity to noise and adversarial attacks. We introduce the weight map layer, a modular layer based on the convolutional layer, that can increase model robustness to noise and adversarial attacks. We conclude our work by a general discussion about the investigated relation between modularity and the addressed problems and potential future research directions