35,033 research outputs found
Adaptive Graph via Multiple Kernel Learning for Nonnegative Matrix Factorization
Nonnegative Matrix Factorization (NMF) has been continuously evolving in
several areas like pattern recognition and information retrieval methods. It
factorizes a matrix into a product of 2 low-rank non-negative matrices that
will define parts-based, and linear representation of nonnegative data.
Recently, Graph regularized NMF (GrNMF) is proposed to find a compact
representation,which uncovers the hidden semantics and simultaneously respects
the intrinsic geometric structure. In GNMF, an affinity graph is constructed
from the original data space to encode the geometrical information. In this
paper, we propose a novel idea which engages a Multiple Kernel Learning
approach into refining the graph structure that reflects the factorization of
the matrix and the new data space. The GrNMF is improved by utilizing the graph
refined by the kernel learning, and then a novel kernel learning method is
introduced under the GrNMF framework. Our approach shows encouraging results of
the proposed algorithm in comparison to the state-of-the-art clustering
algorithms like NMF, GrNMF, SVD etc.Comment: This paper has been withdrawn by the author due to the terrible
writin
Revisiting the Nystrom Method for Improved Large-Scale Machine Learning
We reconsider randomized algorithms for the low-rank approximation of
symmetric positive semi-definite (SPSD) matrices such as Laplacian and kernel
matrices that arise in data analysis and machine learning applications. Our
main results consist of an empirical evaluation of the performance quality and
running time of sampling and projection methods on a diverse suite of SPSD
matrices. Our results highlight complementary aspects of sampling versus
projection methods; they characterize the effects of common data preprocessing
steps on the performance of these algorithms; and they point to important
differences between uniform sampling and nonuniform sampling methods based on
leverage scores. In addition, our empirical results illustrate that existing
theory is so weak that it does not provide even a qualitative guide to
practice. Thus, we complement our empirical results with a suite of worst-case
theoretical bounds for both random sampling and random projection methods.
These bounds are qualitatively superior to existing bounds---e.g. improved
additive-error bounds for spectral and Frobenius norm error and relative-error
bounds for trace norm error---and they point to future directions to make these
algorithms useful in even larger-scale machine learning applications.Comment: 60 pages, 15 color figures; updated proof of Frobenius norm bounds,
added comparison to projection-based low-rank approximations, and an analysis
of the power method applied to SPSD sketche
Similarity Learning via Kernel Preserving Embedding
Data similarity is a key concept in many data-driven applications. Many
algorithms are sensitive to similarity measures. To tackle this fundamental
problem, automatically learning of similarity information from data via
self-expression has been developed and successfully applied in various models,
such as low-rank representation, sparse subspace learning, semi-supervised
learning. However, it just tries to reconstruct the original data and some
valuable information, e.g., the manifold structure, is largely ignored. In this
paper, we argue that it is beneficial to preserve the overall relations when we
extract similarity information. Specifically, we propose a novel similarity
learning framework by minimizing the reconstruction error of kernel matrices,
rather than the reconstruction error of original data adopted by existing work.
Taking the clustering task as an example to evaluate our method, we observe
considerable improvements compared to other state-of-the-art methods. More
importantly, our proposed framework is very general and provides a novel and
fundamental building block for many other similarity-based tasks. Besides, our
proposed kernel preserving opens up a large number of possibilities to embed
high-dimensional data into low-dimensional space.Comment: Published in AAAI 201
Structured Matrix Learning under Arbitrary Entrywise Dependence and Estimation of Markov Transition Kernel
The problem of structured matrix estimation has been studied mostly under
strong noise dependence assumptions. This paper considers a general framework
of noisy low-rank-plus-sparse matrix recovery, where the noise matrix may come
from any joint distribution with arbitrary dependence across entries. We
propose an incoherent-constrained least-square estimator and prove its
tightness both in the sense of deterministic lower bound and matching minimax
risks under various noise distributions. To attain this, we establish a novel
result asserting that the difference between two arbitrary low-rank incoherent
matrices must spread energy out across its entries, in other words cannot be
too sparse, which sheds light on the structure of incoherent low-rank matrices
and may be of independent interest. We then showcase the applications of our
framework to several important statistical machine learning problems. In the
problem of estimating a structured Markov transition kernel, the proposed
method achieves the minimax optimality and the result can be extended to
estimating the conditional mean operator, a crucial component in reinforcement
learning. The applications to multitask regression and structured covariance
estimation are also presented. We propose an alternating minimization algorithm
to approximately solve the potentially hard optimization problem. Numerical
results corroborate the effectiveness of our method which typically converges
in a few steps.Comment: 55 pages, 4 figure
Tensor Decomposition in Multiple Kernel Learning
Modern data processing and analytic tasks often deal with high dimensional matrices or tensors; for example: environmental sensors monitor (time, location, temperature, light) data. For large scale tensors, efficient data representation plays a major role in reducing computational time and finding patterns.
The thesis firstly studies about fundamental matrix, tensor decomposition algorithms and applications, in connection with Tensor Train decomposition algorithm. The second objective is applying the tensor perspective in Multiple Kernel Learning problems, where the stacking of kernels can be seen as a tensor. Decomposition this kind of tensor leads to an efficient factorization approach in finding the best linear combination of kernels through the similarity alignment. Interestingly, thanks to the symmetry of the kernel matrix, a novel decomposition algorithm for multiple kernels is derived for reducing the computational complexity.
In term of applications, this new approach allows the manipulation of large scale multiple kernels problems. For example, with P kernels and n samples, it reduces the memory complexity of O(P^2n^2) to O(P^2r^2+ 2rn) where r < n is the number of low-rank components. This compression is also valuable in pair-wise multiple kernel learning problem which models the relation among pairs of objects and its complexity is in the double scale.
This study proposes AlignF_TT, a kernel alignment algorithm which is based on the novel decomposition algorithm for the tensor of kernels. Regarding the predictive performance, the proposed algorithm can gain an improvement in 18 artificially constructed datasets and achieve comparable performance in 13 real-world datasets in comparison with other multiple kernel learning algorithms. It also reveals that the small number of low-rank components is sufficient for approximating the tensor of kernels
Review on Automatic Face Naming by Learning Discriminative Affinity Matrices from Weakly Labeled Images
Given a set of pictures, wherever every image contains many faces and is related to a number of names within the corresponding caption, the goal of face naming is to give the right name for every face. During this paper, we tend to propose 2 new ways to effectively solve this downside by learning 2 discriminative affinity matrices from these labeled pictures. we tend to first propose a replacement methodology referred to as regular low-rank illustration by effectively utilizing supervised data to be told a low-rank reconstruction constant matrix whereas exploring multiple topological space structures of the information. Specifically, by introducing a specially designed regularizer to the low-rank illustration methodology, we tend to penalise the corresponding reconstruction coefficients associated with the things wherever a face is reconstructed by exploitation face pictures from alternative subjects or by exploitation itself. With the inferred reconstruction constant matrix, a discriminative affinity matrix is often obtained. Moreover, we tend to conjointly develop a replacement distance metric learning methodology referred to as equivocally supervised structural metric learning by exploitation feeble supervised data to hunt a discriminative distance metric. Hence, another discriminative affinity matrix are often obtained exploitation the similarity matrix (i.e., the kernel matrix) supported the Mahalanobis distances of the information. Perceptive that these 2 affinity matrices contain complementary data, we tend to mix those to get a consolidated affinity matrix supported that we tend to develop a replacement reiterative theme to infer the name of every face. Comprehensive experiments demonstrate the effectiveness of our approach. General TermsAffinity matrix, caption-based face naming
- …