17,491 research outputs found
Optimization with Sparsity-Inducing Penalties
Sparse estimation methods are aimed at using or obtaining parsimonious
representations of data or models. They were first dedicated to linear variable
selection but numerous extensions have now emerged such as structured sparsity
or kernel selection. It turns out that many of the related estimation problems
can be cast as convex optimization problems by regularizing the empirical risk
with appropriate non-smooth norms. The goal of this paper is to present from a
general perspective optimization tools and techniques dedicated to such
sparsity-inducing penalties. We cover proximal methods, block-coordinate
descent, reweighted -penalized techniques, working-set and homotopy
methods, as well as non-convex formulations and extensions, and provide an
extensive set of experiments to compare various algorithms from a computational
point of view
Multi-view Metric Learning in Vector-valued Kernel Spaces
We consider the problem of metric learning for multi-view data and present a
novel method for learning within-view as well as between-view metrics in
vector-valued kernel spaces, as a way to capture multi-modal structure of the
data. We formulate two convex optimization problems to jointly learn the metric
and the classifier or regressor in kernel feature spaces. An iterative
three-step multi-view metric learning algorithm is derived from the
optimization problems. In order to scale the computation to large training
sets, a block-wise Nystr{\"o}m approximation of the multi-view kernel matrix is
introduced. We justify our approach theoretically and experimentally, and show
its performance on real-world datasets against relevant state-of-the-art
methods
Learning Multiple Visual Tasks while Discovering their Structure
Multi-task learning is a natural approach for computer vision applications
that require the simultaneous solution of several distinct but related
problems, e.g. object detection, classification, tracking of multiple agents,
or denoising, to name a few. The key idea is that exploring task relatedness
(structure) can lead to improved performances.
In this paper, we propose and study a novel sparse, non-parametric approach
exploiting the theory of Reproducing Kernel Hilbert Spaces for vector-valued
functions. We develop a suitable regularization framework which can be
formulated as a convex optimization problem, and is provably solvable using an
alternating minimization approach. Empirical tests show that the proposed
method compares favorably to state of the art techniques and further allows to
recover interpretable structures, a problem of interest in its own right.Comment: 19 pages, 3 figures, 3 table
- …