88 research outputs found
Random Feature Maps via a Layered Random Projection (LaRP) Framework for Object Classification
The approximation of nonlinear kernels via linear feature maps has recently
gained interest due to their applications in reducing the training and testing
time of kernel-based learning algorithms. Current random projection methods
avoid the curse of dimensionality by embedding the nonlinear feature space into
a low dimensional Euclidean space to create nonlinear kernels. We introduce a
Layered Random Projection (LaRP) framework, where we model the linear kernels
and nonlinearity separately for increased training efficiency. The proposed
LaRP framework was assessed using the MNIST hand-written digits database and
the COIL-100 object database, and showed notable improvement in object
classification performance relative to other state-of-the-art random projection
methods.Comment: 5 page
Compact Random Feature Maps
Kernel approximation using randomized feature maps has recently gained a lot
of interest. In this work, we identify that previous approaches for polynomial
kernel approximation create maps that are rank deficient, and therefore do not
utilize the capacity of the projected feature space effectively. To address
this challenge, we propose compact random feature maps (CRAFTMaps) to
approximate polynomial kernels more concisely and accurately. We prove the
error bounds of CRAFTMaps demonstrating their superior kernel reconstruction
performance compared to the previous approximation schemes. We show how
structured random matrices can be used to efficiently generate CRAFTMaps, and
present a single-pass algorithm using CRAFTMaps to learn non-linear multi-class
classifiers. We present experiments on multiple standard data-sets with
performance competitive with state-of-the-art results.Comment: 9 page
Polynomial Tensor Sketch for Element-wise Function of Low-Rank Matrix
This paper studies how to sketch element-wise functions of low-rank matrices.
Formally, given low-rank matrix A = [Aij] and scalar non-linear function f, we
aim for finding an approximated low-rank representation of the (possibly
high-rank) matrix [f(Aij)]. To this end, we propose an efficient
sketching-based algorithm whose complexity is significantly lower than the
number of entries of A, i.e., it runs without accessing all entries of [f(Aij)]
explicitly. The main idea underlying our method is to combine a polynomial
approximation of f with the existing tensor sketch scheme for approximating
monomials of entries of A. To balance the errors of the two approximation
components in an optimal manner, we propose a novel regression formula to find
polynomial coefficients given A and f. In particular, we utilize a
coreset-based regression with a rigorous approximation guarantee. Finally, we
demonstrate the applicability and superiority of the proposed scheme under
various machine learning tasks
- …