Semi-Supervised Learning for Scalable and Robust Visual Search

Unlike textual document retrieval, searching of visual data is still far from satisfactory. There exist major gaps between the available solutions and practical needs in both accuracy and computational cost. This thesis aims at the development of robust and scalable solutions for visual search and retrieval. Specifically, we investigate two classes of approaches: graph-based semi-supervised learning and hashing techniques. The graph-based approaches are used to improve accuracy, while hashing approaches are used to improve efficiency and cope with large-scale applications. A common theme shared between these two subareas of our work is the focus on semi-supervised learning paradigm, in which a small set of labeled data is complemented with large unlabeled datasets. Graph-based approaches have emerged as methods of choice for general semi-supervised tasks when no parametric information is available about the data distribution. It treats both labeled and unlabeled samples as vertices in a graph and then instantiates pairwise edges between these vertices to capture affinity between the corresponding samples. A quadratic regularization framework has been widely used for label prediction over such graphs. However, most of the existing graph-based semi-supervised learning methods are sensitive to the graph construction process and the initial labels. We propose a new bivariate graph transduction formulation and an efficient solution via an alternating minimization procedure. Based on this bivariate framework, we also develop new methods to filter unreliable and noisy labels. Extensive experiments over diverse benchmark datasets demonstrate the superior performance of our proposed methods. However, graph-based approaches suffer from the critical bottleneck in scalability since graph construction requires a quadratic complexity and the inference procedure costs even more. The widely used graph construction method relies on nearest neighbor search, which is prohibitive for large-scale applications. In addition, most large-scale visual search problems involve handling high-dimensional visual descriptors, thereby causing another challenge in excessive storage requirement. To handle the scalability issue of both computation and storage, the second part of the thesis focuses on efficient techniques for conducting approximate nearest neighbor (ANN) search, which is key to many machine learning algorithms, including graph-based semi-supervised learning and clustering. Specifically, we propose Semi-Supervised Hashing (SSH) methods that leverage semantic similarity over a small set of labeled data while preventing overfitting. We derive a rigorous formulation in which a supervised term minimizes the empirical errors on the labeled data and an unsupervised term provides effective regularization by maximizing variance and independence of individual bits. Experiments on several large datasets demonstrate the clear performance gain over several state-of-the-art methods without significant increase of the computational cost. The main contributions of the thesis include the following. Bivariate graph transduction: a) a bivariate formulation for graph-based semi-supervised learning with an efficient solution by alternating optimization; b) theoretic analysis from the view of graph cut for the bivariate optimization procedure; c) novel applications of the proposed techniques, such as interactive image retrieval, automatic re-ranking for text based image search, and a brain computer interface (BCI) for image retrieval. Semi-supervised hashing: a) a rigorous semi-supervised paradigm for hash functions learning with a tradeoff between empirical fitness on pair-wise label consistence and an information-theoretic regularizer; b) several efficient solutions for deriving semi-supervised hash functions, including an orthogonal solution using eigen-decomposition, a revised strategy for learning non-orthogonal hash functions, a sequential learning algorithm to derive boosted hash functions, and an extension to unsupervised cases by using pseudo labels. Two parts of the thesis - bivariate graph transduction and semi-supervised hashing - are complimentary and can be combined to achieve significant performance improvement in both speed and accuracy. Hash methods can help build sparse graphs in a linear time fashion and greatly reduce the data size, but they lack sufficient accuracy. Graph-based methods provide unique capabilities to handle non-linear data structures with noisy labels but suffer from high computational complexity. The synergistic combination of the two offers great potential for advancing the state-of-the-art in large-scale visual search and many other applications

Hierarchical Subquery Evaluation for Active Learning on a Graph

To train good supervised and semi-supervised object classifiers, it is critical that we not waste the time of the human experts who are providing the training labels. Existing active learning strategies can have uneven performance, being efficient on some datasets but wasteful on others, or inconsistent just between runs on the same dataset. We propose perplexity based graph construction and a new hierarchical subquery evaluation algorithm to combat this variability, and to release the potential of Expected Error Reduction. Under some specific circumstances, Expected Error Reduction has been one of the strongest-performing informativeness criteria for active learning. Until now, it has also been prohibitively costly to compute for sizeable datasets. We demonstrate our highly practical algorithm, comparing it to other active learning measures on classification datasets that vary in sparsity, dimensionality, and size. Our algorithm is consistent over multiple runs and achieves high accuracy, while querying the human expert for labels at a frequency that matches their desired time budget.Comment: CVPR 201

Differential diagnosis of breast cancer using quantitative, label-free and molecular vibrational imaging

We present a label-free, chemically-selective, quantitative imaging strategy to identify breast cancer and differentiate its subtypes using coherent anti-Stokes Raman scattering (CARS) microscopy. Human normal breast tissue, benign proliferative, as well as in situ and invasive carcinomas, were imaged ex vivo. Simply by visualizing cellular and tissue features appearing on CARS images, cancerous lesions can be readily separated from normal tissue and benign proliferative lesion. To further distinguish cancer subtypes, quantitative disease-related features, describing the geometry and distribution of cancer cell nuclei, were extracted and applied to a computerized classification system. The results show that in situ carcinoma was successfully distinguished from invasive carcinoma, while invasive ductal carcinoma (IDC) and invasive lobular carcinoma were also distinguished from each other. Furthermore, 80% of intermediate-grade IDC and 85% of high-grade IDC were correctly distinguished from each other. The proposed quantitative CARS imaging method has the potential to enable rapid diagnosis of breast cancer

Signal processing and graph-based semi-supervised learning-based fault diagnosis for direct online induction motors

In this thesis, fault diagnosis approaches for direct online induction motors are proposed using signal processing and graph-based semi-supervised learning (GSSL). These approaches are developed using experimental data obtained in the lab for two identical 0.25 HP three-phase squirrel-cage induction motors. Various electrical and mechanical single- and multi-faults are applied to each motor during experiments. Three-phase stator currents and three-dimensional vibration signals are recorded simultaneously in each experiment. In this thesis, Power Spectral Density (PSD)-based stator current amplitude spectrum analysis and one-dimensional Complex Continuous Wavelet Transform (CWT)-based stator current time-scale spectrum analysis are employed to detect broken rotor bar (BRB) faults. An effective single- and multi-fault diagnosis approach is developed using GSSL, where discrete wavelet transform (DWT) is applied to extract features from experimental stator current and vibration data. Three GSSL algorithms (Local and global consistency (LGC), Gaussian field and harmonic functions (GFHF), and greedy-gradient max-cut (GGMC)) are adopted and compared in this study. To enable machine learning for untested motor operating conditions, mathematical equations to calculate features for untested conditions are developed using curve fitting and features obtained from experimental data of tested conditions

Shape similarity analysis by self-tuning locally constrained mixed-diffusion

Similarity analysis is a powerful tool for shape matching/retrieval and other computer vision tasks. In the literature, various shape (dis)similarity measures have been introduced. Different measures specialize on different aspects of the data. In this paper, we consider the problem of improving retrieval accuracy by systematically fusing several different measures. To this end, we propose the locally constrained mixeddiffusion method, which partly fuses the given measures into one and propagates on the resulted locally dense data space. Furthermore, we advocate the use of self-adaptive neighborhoods to automatically determine the appropriate size of the neighborhoods in the diffusion process, with which the retrieval performance is comparable to the best manually tuned kNNs. The superiority of our approach is empirically demonstrated on both shape and image datasets. Our approach achieves a score of 100% in the bull’s eye test on the MPEG-7 shape dataset, which is the best reported result to date.Lei Luo, Chunhua Shen, Chunyuan Zhang and Anton van den Henge

Active microscopic cellular image annotation by superposable graph transduction with imbalanced labels

Systematic content screening of cell phenotypes in microscopic images has been shown promising in gene function understanding and drug design. However, manual annotation of cells and images in genome-wide studies is cost prohibitive. In this paper, we propose a highly efficient active annotation framework, in which a small amount of expert input is leveraged to rapidly and effectively infer the labels over the remaining unlabeled data. We formulate this as a graph based transductive learning problem and develop a novel method for label propagation. Specifically, a label regularizer method is proposed to handle the important label imbalance issue, typically seen in the cellular image screening applications. We also design a new scheme which breaks the graph into linear superposition of contributions from individual labeled samples. We take advantage of such a superposable representation to achieve fast annotation in an interactive setting. Extensive evaluations over toy data and realistic cellular images confirm the superiority of the proposed method over existing alternatives. 1

Clinical cancer diagnosis using optical fiber-delivered coherent anti-stokes ramon scattering microscopy

This thesis describes the development of a combined label-free imaging and analytical strategy for intraoperative characterization of cancer lesions using the coherent anti-Stokes Raman scattering imaging (CARS) technique. A cell morphology-based analytical platform is developed to characterize CARS images and, hence, provide diagnostic information using disease-related pathology features. This strategy is validated for three different applications, including margin detection for radical prostatectomy, differential diagnosis of lung cancer, as well as detection and differentiation of breast cancer subtypes for in situ analysis of margin status during lumpectomy. As the major contribution of this thesis, the developed analytical strategy shows high accuracy and specificity for all three diseases and thus has introduced the CARS imaging technique into the field of human cancer diagnosis, which holds substantial potential for clinical translations. In addition, I have contributed a project aimed at miniaturizing the CARS imaging device into a microendoscope setup through a fiber-delivery strategy. A four-wave-mixing (FWM) background signal, which is caused by simultaneous delivery of the two CARS-generating excitation laser beams, is initially identified. A polarization-based strategy is then introduced and tested for suppression of this FWM noise. The approach shows effective suppression of the FWM signal, both on microscopic and prototype endoscopic setups, indicating the potential of developing a novel microendoscope with a compatible size for clinical use. These positive results show promise for the development of an all-fiber-based, label-free imaging and analytical platform for minimally invasive detection and diagnosis of cancers during surgery or surgical-biopsy, thus improving surgical outcomes and reducing patients' suffering

Rôle de la paroi végétale dans l'interaction entre Arabidopsis thaliana et Ralstonia solanacearum : criblage de mutants « paroi » et caractérisation fine de la résistance accrue du mutant walls are thin 1 (wat1)

De part sa localisation à l'interface entre la cellule et son environnement, la paroi végétale joue un rôle clé lors des interactions avec des agents pathogènes. Au cours de ma thèse, l'importance de la paroi végétale lors de l'infection d'Arabidopsis par la bactérie Ralstonia solanacearum a été étudiée. Pour cela, une analyse immunocytologique couplée à une approche bioinformatique a permis d'identifier les modifications pariétales induites en réponse à l'infection et de les corréler à l'arsenal enzymatique existant chez la bactérie. En parallèle, le crible de mutants ''paroi'' d'Arabidopsis testés pour leur sensibilité à différents agents pathogènes a contribué à ouvrir de nouvelles pistes pour mieux comprendre le rôle de la paroi dans les réponses de défense puisque 28 mutants ont une sensibilité modifiée. L'essentiel de mes recherches a porté sur la caractérisation de wat1 (walls are thin 1), un mutant d'Arabidopsis présentant une résistance accrue à R. solanacearum. Par des approches génétique, transcriptomique et métabolomique, nous avons pu définir que wat1 présente une immunité vasculaire et que les mécanismes de résistance impliqueraient une perturbation dans le " crosstalk " entre les métabolismes de l'auxine, des glucosinolates indoliques et de l'acide salicylique au niveau du système racinaire plutôt que des modifications pariétales stricto sensu.Due to its location at the interface between the cell and its environment, the plant cell wall plays a key role in interactions with pathogens. During this project, the importance of plant cell walls during infection of Arabidopsis by the bacterium Ralstonia solanacearum has been studied. First, an immunocytological analysis coupled with a bioinformatic approach allowed us to identify cell wall modifications in response to infection and to correlate them to the cell wall-degrading enzymatic arsenal present in the bacteria. In parallel, the screening of cell wall mutants of Arabidopsis for susceptibility to different pathogens led to the identification of twenty eight mutants with altered sensitivity and, as a result, will open new avenues for understanding the role of the wall in defense responses. Much of my PhD research has focused on the characterization of wat1 (walls are thin 1), an Arabidopsis mutant with increased resistance to R. solanacearum. Through combined genetic, transcriptomic, and metabolomic approaches, we show that wat1 exhibits vascular immunity, most likely resulting from altered crosstalk between auxin, indole glucosinolate and salicylic acid metabolism in roots rather than in cell wall modifications per se