Deep learning with very few and no labels

Deep neural networks have achieved remarkable performance in many computer vision applications such as image classification, object detection, instance segmentation, image retrieval, and person re-identification. However, to achieve the desired performance, deep neural networks often need a tremendously large set of labeled training samples to learn its huge network model. Labeling a large dataset is labor-intensive, time-consuming, and sometimes requiring expert knowledge. In this research, we study the following important question: how to train deep neural networks with very few or even no labeled samples? This leads to our research tasks in the following two major areas: semi-supervised and unsupervised learning. Specifically, for semi-supervised learning, we developed two major approaches. The first one is the Snowball approach which learns a deep neural network from very few samples based on iterative model evolution and confident sample discovery. The second one is the learned model composition approach which composes more efficient master networks from student models of past iterations through a network learning process. Critical sample discovery is developed to discover new critical unlabeled samples near the model decision boundary and provide the master model with lookahead access to these samples to enhance its guidance capability. For unsupervised learning, we have explored two major ideas. The first idea is transformed attention consistency where the network is learned based on selfsupervision information across images instead of within one single image. The second one is spatial assembly networks for image representation learning. We introduce a new learnable module, called spatial assembly network (SAN), which performs a learned re-organization and assembly of feature points and improves the network capabilities in handling spatial variations and structural changes of the image scene. Our experimental results on benchmark datasets demonstrate that our proposed methods have significantly improved the state-of-the-art in semi-supervised and unsupervised learning, outperforming existing methods by large margins.Includes bibliographical references

2014 Update of the Alzheimer's Disease Neuroimaging Initiative: A review of papers published since its inception

The Alzheimer's Disease Neuroimaging Initiative (ADNI) is an ongoing, longitudinal, multicenter study designed to develop clinical, imaging, genetic, and biochemical biomarkers for the early detection and tracking of Alzheimer's disease (AD). The initial study, ADNI-1, enrolled 400 subjects with early mild cognitive impairment (MCI), 200 with early AD, and 200 cognitively normal elderly controls. ADNI-1 was extended by a 2-year Grand Opportunities grant in 2009 and by a competitive renewal, ADNI-2, which enrolled an additional 550 participants and will run until 2015. This article reviews all papers published since the inception of the initiative and summarizes the results to the end of 2013. The major accomplishments of ADNI have been as follows: (1) the development of standardized methods for clinical tests, magnetic resonance imaging (MRI), positron emission tomography (PET), and cerebrospinal fluid (CSF) biomarkers in a multicenter setting; (2) elucidation of the patterns and rates of change of imaging and CSF biomarker measurements in control subjects, MCI patients, and AD patients. CSF biomarkers are largely consistent with disease trajectories predicted by β-amyloid cascade (Hardy, J Alzheimer's Dis 2006;9(Suppl 3):151-3) and tau-mediated neurodegeneration hypotheses for AD, whereas brain atrophy and hypometabolism levels show predicted patterns but exhibit differing rates of change depending on region and disease severity; (3) the assessment of alternative methods of diagnostic categorization. Currently, the best classifiers select and combine optimum features from multiple modalities, including MRI, [(18)F]-fluorodeoxyglucose-PET, amyloid PET, CSF biomarkers, and clinical tests; (4) the development of blood biomarkers for AD as potentially noninvasive and low-cost alternatives to CSF biomarkers for AD diagnosis and the assessment of α-syn as an additional biomarker; (5) the development of methods for the early detection of AD. CSF biomarkers, β-amyloid 42 and tau, as well as amyloid PET may reflect the earliest steps in AD pathology in mildly symptomatic or even nonsymptomatic subjects and are leading candidates for the detection of AD in its preclinical stages; (6) the improvement of clinical trial efficiency through the identification of subjects most likely to undergo imminent future clinical decline and the use of more sensitive outcome measures to reduce sample sizes. Multimodal methods incorporating APOE status and longitudinal MRI proved most highly predictive of future decline. Refinements of clinical tests used as outcome measures such as clinical dementia rating-sum of boxes further reduced sample sizes; (7) the pioneering of genome-wide association studies that leverage quantitative imaging and biomarker phenotypes, including longitudinal data, to confirm recently identified loci, CR1, CLU, and PICALM and to identify novel AD risk loci; (8) worldwide impact through the establishment of ADNI-like programs in Japan, Australia, Argentina, Taiwan, China, Korea, Europe, and Italy; (9) understanding the biology and pathobiology of normal aging, MCI, and AD through integration of ADNI biomarker and clinical data to stimulate research that will resolve controversies about competing hypotheses on the etiopathogenesis of AD, thereby advancing efforts to find disease-modifying drugs for AD; and (10) the establishment of infrastructure to allow sharing of all raw and processed data without embargo to interested scientific investigators throughout the world

Metodología de reducción de dimensión de tipo espectral con representación interactiva de datos

La reducción de dimensión (RD) es una metodología utilizada en muchos campos ligados al procesamiento de datos, y puede representar una etapa de preproceso o ser un elemento esencial para la representación y clasificación de datos. El objetivo principal de la RD es obtener una nueva representación de los datos originales en un espacio de menor dimensión, de forma que se produzca información más depurada, reduzca el tiempo del procesado subsecuente o genere representaciones visuales inteligibles para el ser humano. Los métodos recientes y más sofisticados de RD exploran la topología de los datos, entre estos se encuentran los enfoques de tipo espectral. Particularmente, los métodos espectrales son altamente versátiles y han comprobado ser una buena alternativa para diversas aplicaciones. Estos métodos no permiten manipular directamente sus parámetros, y, por tanto, el usuario final queda sometido a las representaciones visuales resultantes, que en muchos de los casos requieren de un experto para su análisis, puesto que no se ajustan a las necesidades y los requerimiento del usuario. En este sentido, se genera implícitamente un incremento en tiempo y trabajo en la inspección visual, realizada como el último paso del análisis de datos. Una de las formas de generar representaciones más adecuadas para el usuario y que permiten deducir un mejor conocimiento es integrar la inteligencia natural del ser humano con la inteligencia de la máquina. Para esto, es necesario integrar propiedades de la visualización de información (VI), como la interactividad y la controlabilidad, de forma que el usuario tenga la facultad de variar los parámetros de los métodos de RD hasta obtener una representación que se adapte a sus necesidades. Los métodos espectrales requieren realizar un proceso de descomposición en valores y vectores propios, el cual suele presentar un costo computacional elevado, y, por tanto, resulta difícil la tarea de obtener una integración usuario-máquina más dinámica e interactiva. Por lo anterior, para el diseño de un sistema interactivo de VI basado en métodos espectrales de RD es necesario plantear una estrategia para disminuir el coste computacional requerido en el cálculo de los vectores y valores propios. En este trabajo de grado de maestría se propone una metodología de RD espectral con bajo costo computacional para la representación interactiva de datos. Para este fin, se propone utilizar submatrices localmente lineales como aproximación de una matriz de afinidad. Además, se propone un modelo interactivo que permita al usuario obtener una representación visual dinámica de los datos mediante una mezcla ponderada. Esto permite integrar la inteligencia natural con la computacional para la representación de datos de forma interactiva, dinámica y a bajo costo computacionalAbstract: Dimensionality reduction (DR) is a methodology used in many fields linked to data processing, and may represent a preprocessing stage or be an essential element for the representation and classification of data. The main objective of DR is to obtain a new representation of the original data in a space of smaller dimension, such that more refined information is produced, as well as the time of the subsequent processing is decreased and/or visual representations more intelligible for human beings are generated. The recent and more sophisticated DR methods are those that explore the topology of the data, being the spectral approaches. In particular, the spectral methods are highly versatile and have proven to be a good alternative for various applications. In terms of information visualization (IV), DR methods have been widely used to generate visual representations generated by algorithms that work under pre-established criteria. These methods do not allow direct manipulation of their parameters, and, therefore, the end user is subject to the resulting visual representations, which in many cases require an expert for analysis. In this sense, an increase in time and work is implicitly generated in the visual inspection, in addition to the costs in the process of determining information useful to the user, which represents the ultimate goal of data processing. In addition, these representations do not conform to the needs and requirements of the user. To generate more appropriate representations for the user and that allows us to deduce a better knowledge is to integrate the natural intelligence of the human being with the intelligence of the machine. To this purpose, it is necessary to integrate properties of IV, such as interactivity and controllability, so that the user has the ability to vary the parameters of the DE methods until obtaining a representation that suits its needs. The spectral DR methods involve the calculation of an eigenvalue and eigenvector decomposition, which is usually high-computational-cost demanding, and, therefore, the task of obtaining a more dynamic and interactive user-machine integration is difficult. Therefore, for the design of an interactive IV system based on DR spectral methods, it is necessary to propose a strategy to reduce the computational cost required in the calculation of eigenvectors and eigenvalues. In this work, a methodology of spectral dimensionality reduction involving low-computational cost for the interactive representation of data is proposed. For this purpose, it is proposed to use locally linear submatrices and spectral embedding. This allows integrating natural intelligence with computational intelligence for the representation of data interactively, dynamically and at low computational cost. Additionally, an interactive model is proposed that allows the user to dynamically visualize the data through a weighted mixtureMaestrí