1,648 research outputs found
High-Dimensional Screening Using Multiple Grouping of Variables
Screening is the problem of finding a superset of the set of non-zero entries
in an unknown p-dimensional vector \beta* given n noisy observations.
Naturally, we want this superset to be as small as possible. We propose a novel
framework for screening, which we refer to as Multiple Grouping (MuG), that
groups variables, performs variable selection over the groups, and repeats this
process multiple number of times to estimate a sequence of sets that contains
the non-zero entries in \beta*. Screening is done by taking an intersection of
all these estimated sets. The MuG framework can be used in conjunction with any
group based variable selection algorithm. In the high-dimensional setting,
where p >> n, we show that when MuG is used with the group Lasso estimator,
screening can be consistently performed without using any tuning parameter. Our
numerical simulations clearly show the merits of using the MuG framework in
practice.Comment: This paper will appear in the IEEE Transactions on Signal Processing.
See http://www.ima.umn.edu/~dvats/MuGScreening.html for more detail
Screening Rules for Convex Problems
We propose a new framework for deriving screening rules for convex
optimization problems. Our approach covers a large class of constrained and
penalized optimization formulations, and works in two steps. First, given any
approximate point, the structure of the objective function and the duality gap
is used to gather information on the optimal solution. In the second step, this
information is used to produce screening rules, i.e. safely identifying
unimportant weight variables of the optimal solution. Our general framework
leads to a large variety of useful existing as well as new screening rules for
many applications. For example, we provide new screening rules for general
simplex and -constrained problems, Elastic Net, squared-loss Support
Vector Machines, minimum enclosing ball, as well as structured norm regularized
problems, such as group lasso
GAP Safe screening rules for sparse multi-task and multi-class models
High dimensional regression benefits from sparsity promoting regularizations.
Screening rules leverage the known sparsity of the solution by ignoring some
variables in the optimization, hence speeding up solvers. When the procedure is
proven not to discard features wrongly the rules are said to be \emph{safe}. In
this paper we derive new safe rules for generalized linear models regularized
with and norms. The rules are based on duality gap
computations and spherical safe regions whose diameters converge to zero. This
allows to discard safely more variables, in particular for low regularization
parameters. The GAP Safe rule can cope with any iterative solver and we
illustrate its performance on coordinate descent for multi-task Lasso, binary
and multinomial logistic regression, demonstrating significant speed ups on all
tested datasets with respect to previous safe rules.Comment: in Proceedings of the 29-th Conference on Neural Information
Processing Systems (NIPS), 201
- …