Sequential Dimensionality Reduction for Extracting Localized Features

Linear dimensionality reduction techniques are powerful tools for image analysis as they allow the identification of important features in a data set. In particular, nonnegative matrix factorization (NMF) has become very popular as it is able to extract sparse, localized and easily interpretable features by imposing an additive combination of nonnegative basis elements. Nonnegative matrix underapproximation (NMU) is a closely related technique that has the advantage to identify features sequentially. In this paper, we propose a variant of NMU that is particularly well suited for image analysis as it incorporates the spatial information, that is, it takes into account the fact that neighboring pixels are more likely to be contained in the same features, and favors the extraction of localized features by looking for sparse basis elements. We show that our new approach competes favorably with comparable state-of-the-art techniques on synthetic, facial and hyperspectral image data sets.Comment: 24 pages, 12 figures. New numerical experiments on synthetic data sets, discussion about the convergenc

Using Underapproximations for Sparse Nonnegative Matrix Factorization

Nonnegative Matrix Factorization consists in (approximately) factorizing a nonnegative data matrix by the product of two low-rank nonnegative matrices. It has been successfully applied as a data analysis technique in numerous domains, e.g., text mining, image processing, microarray data analysis, collaborative filtering, etc. We introduce a novel approach to solve NMF problems, based on the use of an underapproximation technique, and show its effectiveness to obtain sparse solutions. This approach, based on Lagrangian relaxation, allows the resolution of NMF problems in a recursive fashion. We also prove that the underapproximation problem is NP-hard for any fixed factorization rank, using a reduction of the maximum edge biclique problem in bipartite graphs. We test two variants of our underapproximation approach on several standard image datasets and show that they provide sparse part-based representations with low reconstruction error. Our results are comparable and sometimes superior to those obtained by two standard Sparse Nonnegative Matrix Factorization techniques.Comment: Version 2 removed the section about convex reformulations, which was not central to the development of our main results; added material to the introduction; added a review of previous related work (section 2.3); completely rewritten the last part (section 4) to provide extensive numerical results supporting our claims. Accepted in J. of Pattern Recognitio

Two Algorithms for Orthogonal Nonnegative Matrix Factorization with Application to Clustering

Approximate matrix factorization techniques with both nonnegativity and orthogonality constraints, referred to as orthogonal nonnegative matrix factorization (ONMF), have been recently introduced and shown to work remarkably well for clustering tasks such as document classification. In this paper, we introduce two new methods to solve ONMF. First, we show athematical equivalence between ONMF and a weighted variant of spherical k-means, from which we derive our first method, a simple EM-like algorithm. This also allows us to determine when ONMF should be preferred to k-means and spherical k-means. Our second method is based on an augmented Lagrangian approach. Standard ONMF algorithms typically enforce nonnegativity for their iterates while trying to achieve orthogonality at the limit (e.g., using a proper penalization term or a suitably chosen search direction). Our method works the opposite way: orthogonality is strictly imposed at each step while nonnegativity is asymptotically obtained, using a quadratic penalty. Finally, we show that the two proposed approaches compare favorably with standard ONMF algorithms on synthetic, text and image data sets.Comment: 17 pages, 8 figures. New numerical experiments (document and synthetic data sets

Using machine learning on the sources of retinal images for diagnosis by proxy of diabetes mellitus and diabetic retinopathy

In current research in ophthalmology, images of the vascular system in the human retina are used as exploratory proxies for pathologies affecting different organs. This thesis addresses the analysis, using machine learning and computer vision techniques, of retinal images acquired with different techniques (Fundus retinographies, optical coherence tomography and optical coherence tomography angiography), with the objective of using them to assist diagnostic decision making in diabetes mellitus and diabetic retinopathy. This thesis explores the use of matrix factorization-based source extraction techniques, as the basis to transform the retinal images for classification. The proposed approach consists on preprocessing the images to enable the learning of an unsupervised parts-based representation prior to the classification. As a result of the use of interpretable models, with this approach we unveiled an important bias in the data. After correcting for the bias, promising results were still obtained which merit for further exploration

On Restricted Nonnegative Matrix Factorization

Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative $n \times m$ matrix $M$ into a product of a nonnegative $n \times d$ matrix $W$ and a nonnegative $d \times m$ matrix $H$. Restricted NMF requires in addition that the column spaces of $M$ and $W$ coincide. Finding the minimal inner dimension $d$ is known to be NP-hard, both for NMF and restricted NMF. We show that restricted NMF is closely related to a question about the nature of minimal probabilistic automata, posed by Paz in his seminal 1971 textbook. We use this connection to answer Paz's question negatively, thus falsifying a positive answer claimed in 1974. Furthermore, we investigate whether a rational matrix $M$ always has a restricted NMF of minimal inner dimension whose factors $W$ and $H$ are also rational. We show that this holds for matrices $M$ of rank at most $3$ and we exhibit a rank-$4$ matrix for which $W$ and $H$ require irrational entries.Comment: Full version of an ICALP'16 pape

A Block Coordinate Descent-based Projected Gradient Algorithm for Orthogonal Non-negative Matrix Factorization

This article utilizes the projected gradient method (PG) for a non-negative matrix factorization problem (NMF), where one or both matrix factors must have orthonormal columns or rows. We penalise the orthonormality constraints and apply the PG method via a block coordinate descent approach. This means that at a certain time one matrix factor is fixed and the other is updated by moving along the steepest descent direction computed from the penalised objective function and projecting onto the space of non-negative matrices. Our method is tested on two sets of synthetic data for various values of penalty parameters. The performance is compared to the well-known multiplicative update (MU) method from Ding (2006), and with a modified global convergent variant of the MU algorithm recently proposed by Mirzal (2014). We provide extensive numerical results coupled with appropriate visualizations, which demonstrate that our method is very competitive and usually outperforms the other two methods

A hybrid approach for stain normalisation in digital histopathological images

Stain in-homogeneity adversely affects segmentation and quantifi-cation of tissues in histology images. Stain normalisation techniques have been used to standardise the appearance of images. However, most the available stain normalisation techniques only work on a particular kind of stain images. In addition, some of these techniques fail to utilise both the spatial and tex-tural information in histology images, leading to image tissue distortion. In this paper, a hybrid approach has been developed, based on an octree colour quantisation algorithm combined with the Beer-Lambert law, a modified blind source separation algorithm, and a modified colour transfer approach. The hybrid method consists of two stages the stain separation stage and colour transfer stage. An octree colour quantisation algorithm combined with Beer-Lambert law, and a modified blind source separation algorithm are used during the stain separation stage to computationally estimate the amount of stain in an histology image based on its chromatic and luminous response. A modified colour transfer algorithm is used during the colour transfer stage to minimise the effect of varying staining and illumination. The hybrid method addresses the colour variation problem in both H&DAB (Haemotoxylin and Diaminoben-zidine) and H&E (Haemotoxylin and Eosin) stain images. The stain normali-sation method is validated against ground truth data. It is widely known that the Beer-Lambert law applies to only stains (such as haematoxylin, eosin) that absorb light. We demonstrate that the Beer-Lambert law applies is applicable to images containing a DAB stain. Better stain normalisation results are obtained in both H&E and H&DAB images