Primal-Dual Algorithms for Non-negative Matrix Factorization with the Kullback-Leibler Divergence

Non-negative matrix factorization (NMF) approximates a given matrix as a product of two non-negative matrices. Multiplicative algorithms deliver reliable results, but they show slow convergence for high-dimensional data and may be stuck away from local minima. Gradient descent methods have better behavior, but only apply to smooth losses such as the least-squares loss. In this article, we propose a first-order primal-dual algorithm for non-negative decomposition problems (where one factor is fixed) with the KL divergence, based on the Chambolle-Pock algorithm. All required computations may be obtained in closed form and we provide an efficient heuristic way to select step-sizes. By using alternating optimization, our algorithm readily extends to NMF and, on synthetic examples, face recognition or music source separation datasets, it is either faster than existing algorithms, or leads to improved local optima, or both

Non-negative matrix factorization methods for face recognition under extreme lighting variations

Abstract-Face recognition task is of primary interest in many computer vision applications, including access control for security systems, forensic or surveillance. Most commercial biometric systems based on face recognition are claimed to perform satisfactory when the enrollment and testing process takes place under controlled environmental conditions such as constant illumination, constant pose scale, non-occluded faces or frontal view. More or less deviation from those conditions might lead to poor recognition performances or even recognition system's failure when a test identity has to be recognized under new modified testing conditions. Three non-negative matrix factorization (NMF) methods, namely, the standard one, the local NMF (LNMF) and the discriminant NMF (DNMF) are employed in this paper where their robustness against extreme lighting variations are tested for the face recognition task. Principal Component Analysis (PCA) method was also chosen as baseline. Experiments revealed that the best recognition performance is obtained with NMF, followed by DNMF and LNMF

New SVD based initialization strategy for Non-negative Matrix Factorization

There are two problems need to be dealt with for Non-negative Matrix Factorization (NMF): choose a suitable rank of the factorization and provide a good initialization method for NMF algorithms. This paper aims to solve these two problems using Singular Value Decomposition (SVD). At first we extract the number of main components as the rank, actually this method is inspired from [1, 2]. Second, we use the singular value and its vectors to initialize NMF algorithm. In 2008, Boutsidis and Gollopoulos [3] provided the method titled NNDSVD to enhance initialization of NMF algorithms. They extracted the positive section and respective singular triplet information of the unit matrices {C(j)}k j=1 which were obtained from singular vector pairs. This strategy aims to use positive section to cope with negative elements of the singular vectors, but in experiments we found that even replacing negative elements by their absolute values could get better results than NNDSVD. Hence, we give another method based SVD to fulfil initialization for NMF algorithms (SVD-NMF). Numerical experiments on two face databases ORL and YALE [16, 17] show that our method is better than NNDSVD

A deep matrix factorization method for learning attribute representations

Semi-Non-negative Matrix Factorization is a technique that learns a low-dimensional representation of a dataset that lends itself to a clustering interpretation. It is possible that the mapping between this new representation and our original data matrix contains rather complex hierarchical information with implicit lower-level hidden attributes, that classical one level clustering methodologies can not interpret. In this work we propose a novel model, Deep Semi-NMF, that is able to learn such hidden representations that allow themselves to an interpretation of clustering according to different, unknown attributes of a given dataset. We also present a semi-supervised version of the algorithm, named Deep WSF, that allows the use of (partial) prior information for each of the known attributes of a dataset, that allows the model to be used on datasets with mixed attribute knowledge. Finally, we show that our models are able to learn low-dimensional representations that are better suited for clustering, but also classification, outperforming Semi-Non-negative Matrix Factorization, but also other state-of-the-art methodologies variants.Comment: Submitted to TPAMI (16-Mar-2015

Adaptive Graph via Multiple Kernel Learning for Nonnegative Matrix Factorization

Nonnegative Matrix Factorization (NMF) has been continuously evolving in several areas like pattern recognition and information retrieval methods. It factorizes a matrix into a product of 2 low-rank non-negative matrices that will define parts-based, and linear representation of nonnegative data. Recently, Graph regularized NMF (GrNMF) is proposed to find a compact representation,which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure. In GNMF, an affinity graph is constructed from the original data space to encode the geometrical information. In this paper, we propose a novel idea which engages a Multiple Kernel Learning approach into refining the graph structure that reflects the factorization of the matrix and the new data space. The GrNMF is improved by utilizing the graph refined by the kernel learning, and then a novel kernel learning method is introduced under the GrNMF framework. Our approach shows encouraging results of the proposed algorithm in comparison to the state-of-the-art clustering algorithms like NMF, GrNMF, SVD etc.Comment: This paper has been withdrawn by the author due to the terrible writin