22,249 research outputs found
The solution path of the generalized lasso
We present a path algorithm for the generalized lasso problem. This problem
penalizes the norm of a matrix D times the coefficient vector, and has
a wide range of applications, dictated by the choice of D. Our algorithm is
based on solving the dual of the generalized lasso, which greatly facilitates
computation of the path. For (the usual lasso), we draw a connection
between our approach and the well-known LARS algorithm. For an arbitrary D, we
derive an unbiased estimate of the degrees of freedom of the generalized lasso
fit. This estimate turns out to be quite intuitive in many applications.Comment: Published in at http://dx.doi.org/10.1214/11-AOS878 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
A Path Algorithm for Constrained Estimation
Many least squares problems involve affine equality and inequality
constraints. Although there are variety of methods for solving such problems,
most statisticians find constrained estimation challenging. The current paper
proposes a new path following algorithm for quadratic programming based on
exact penalization. Similar penalties arise in regularization in model
selection. Classical penalty methods solve a sequence of unconstrained problems
that put greater and greater stress on meeting the constraints. In the limit as
the penalty constant tends to , one recovers the constrained solution.
In the exact penalty method, squared penalties are replaced by absolute value
penalties, and the solution is recovered for a finite value of the penalty
constant. The exact path following method starts at the unconstrained solution
and follows the solution path as the penalty constant increases. In the
process, the solution path hits, slides along, and exits from the various
constraints. Path following in lasso penalized regression, in contrast, starts
with a large value of the penalty constant and works its way downward. In both
settings, inspection of the entire solution path is revealing. Just as with the
lasso and generalized lasso, it is possible to plot the effective degrees of
freedom along the solution path. For a strictly convex quadratic program, the
exact penalty algorithm can be framed entirely in terms of the sweep operator
of regression analysis. A few well chosen examples illustrate the mechanics and
potential of path following.Comment: 26 pages, 5 figure
- …