6,090 research outputs found
Exploring Large Feature Spaces with Hierarchical Multiple Kernel Learning
For supervised and unsupervised learning, positive definite kernels allow to
use large and potentially infinite dimensional feature spaces with a
computational cost that only depends on the number of observations. This is
usually done through the penalization of predictor functions by Euclidean or
Hilbertian norms. In this paper, we explore penalizing by sparsity-inducing
norms such as the l1-norm or the block l1-norm. We assume that the kernel
decomposes into a large sum of individual basis kernels which can be embedded
in a directed acyclic graph; we show that it is then possible to perform kernel
selection through a hierarchical multiple kernel learning framework, in
polynomial time in the number of selected kernels. This framework is naturally
applied to non linear variable selection; our extensive simulations on
synthetic datasets and datasets from the UCI repository show that efficiently
exploring the large feature space through sparsity-inducing norms leads to
state-of-the-art predictive performance
Network Flow Algorithms for Structured Sparsity
We consider a class of learning problems that involve a structured
sparsity-inducing norm defined as the sum of -norms over groups of
variables. Whereas a lot of effort has been put in developing fast optimization
methods when the groups are disjoint or embedded in a specific hierarchical
structure, we address here the case of general overlapping groups. To this end,
we show that the corresponding optimization problem is related to network flow
optimization. More precisely, the proximal problem associated with the norm we
consider is dual to a quadratic min-cost flow problem. We propose an efficient
procedure which computes its solution exactly in polynomial time. Our algorithm
scales up to millions of variables, and opens up a whole new range of
applications for structured sparse models. We present several experiments on
image and video data, demonstrating the applicability and scalability of our
approach for various problems.Comment: accepted for publication in Adv. Neural Information Processing
Systems, 201
Controller Synthesis for Discrete-Time Polynomial Systems via Occupation Measures
In this paper, we design nonlinear state feedback controllers for
discrete-time polynomial dynamical systems via the occupation measure approach.
We propose the discrete-time controlled Liouville equation, and use it to
formulate the controller synthesis problem as an infinite-dimensional linear
programming problem on measures, which is then relaxed as finite-dimensional
semidefinite programming problems on moments of measures and their duals on
sums-of-squares polynomials. Nonlinear controllers can be extracted from the
solutions to the relaxed problems. The advantage of the occupation measure
approach is that we solve convex problems instead of generally non-convex
problems, and the computational complexity is polynomial in the state and input
dimensions, and hence the approach is more scalable. In addition, we show that
the approach can be applied to over-approximating the backward reachable set of
discrete-time autonomous polynomial systems and the controllable set of
discrete-time polynomial systems under known state feedback control laws. We
illustrate our approach on several dynamical systems
- …