34,176 research outputs found
Random Feature-based Online Multi-kernel Learning in Environments with Unknown Dynamics
Kernel-based methods exhibit well-documented performance in various nonlinear
learning tasks. Most of them rely on a preselected kernel, whose prudent choice
presumes task-specific prior information. Especially when the latter is not
available, multi-kernel learning has gained popularity thanks to its
flexibility in choosing kernels from a prescribed kernel dictionary. Leveraging
the random feature approximation and its recent orthogonality-promoting
variant, the present contribution develops a scalable multi-kernel learning
scheme (termed Raker) to obtain the sought nonlinear learning function `on the
fly,' first for static environments. To further boost performance in dynamic
environments, an adaptive multi-kernel learning scheme (termed AdaRaker) is
developed. AdaRaker accounts not only for data-driven learning of kernel
combination, but also for the unknown dynamics. Performance is analyzed in
terms of both static and dynamic regrets. AdaRaker is uniquely capable of
tracking nonlinear learning functions in environments with unknown dynamics,
and with with analytic performance guarantees. Tests with synthetic and real
datasets are carried out to showcase the effectiveness of the novel algorithms.Comment: 36 page
Confident Kernel Sparse Coding and Dictionary Learning
In recent years, kernel-based sparse coding (K-SRC) has received particular
attention due to its efficient representation of nonlinear data structures in
the feature space. Nevertheless, the existing K-SRC methods suffer from the
lack of consistency between their training and test optimization frameworks. In
this work, we propose a novel confident K-SRC and dictionary learning algorithm
(CKSC) which focuses on the discriminative reconstruction of the data based on
its representation in the kernel space. CKSC focuses on reconstructing each
data sample via weighted contributions which are confident in its corresponding
class of data. We employ novel discriminative terms to apply this scheme to
both training and test frameworks in our algorithm. This specific design
increases the consistency of these optimization frameworks and improves the
discriminative performance in the recall phase. In addition, CKSC directly
employs the supervised information in its dictionary learning framework to
enhance the discriminative structure of the dictionary. For empirical
evaluations, we implement our CKSC algorithm on multivariate time-series
benchmarks such as DynTex++ and UTKinect. Our claims regarding the superior
performance of the proposed algorithm are justified throughout comparing its
classification results to the state-of-the-art K-SRC algorithms.Comment: 10 pages, ICDM 2018 conferenc
Reduced Kernel Dictionary Learning
In this paper we present new algorithms for training reduced-size nonlinear
representations in the Kernel Dictionary Learning (KDL) problem. Standard KDL
has the drawback of a large size of the kernel matrix when the data set is
large. There are several ways of reducing the kernel size, notably Nystr\"om
sampling. We propose here a method more in the spirit of dictionary learning,
where the kernel vectors are obtained with a trained sparse representation of
the input signals. Moreover, we optimize directly the kernel vectors in the KDL
process, using gradient descent steps. We show with three data sets that our
algorithms are able to provide better representations, despite using a small
number of kernel vectors, and also decrease the execution time with respect to
KDL
- …