1,523 research outputs found
The Well Tempered Lasso
We study the complexity of the entire regularization path for least squares
regression with 1-norm penalty, known as the Lasso. Every regression parameter
in the Lasso changes linearly as a function of the regularization value. The
number of changes is regarded as the Lasso's complexity. Experimental results
using exact path following exhibit polynomial complexity of the Lasso in the
problem size. Alas, the path complexity of the Lasso on artificially designed
regression problems is exponential.
We use smoothed analysis as a mechanism for bridging the gap between worst
case settings and the de facto low complexity. Our analysis assumes that the
observed data has a tiny amount of intrinsic noise. We then prove that the
Lasso's complexity is polynomial in the problem size. While building upon the
seminal work of Spielman and Teng on smoothed complexity, our analysis is
morally different as it is divorced from specific path following algorithms. We
verify the validity of our analysis in experiments with both worst case
settings and real datasets. The empirical results we obtain closely match our
analysis.Comment: 16 pages, 2 figure
A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors
Variable selection techniques have become increasingly popular amongst
statisticians due to an increased number of regression and classification
applications involving high-dimensional data where we expect some predictors to
be unimportant. In this context, Bayesian variable selection techniques
involving Markov chain Monte Carlo exploration of the posterior distribution
over models can be prohibitively computationally expensive and so there has
been attention paid to quasi-Bayesian approaches such as maximum a posteriori
(MAP) estimation using priors that induce sparsity in such estimates. We focus
on this latter approach, expanding on the hierarchies proposed to date to
provide a Bayesian interpretation and generalization of state-of-the-art
penalized optimization approaches and providing simultaneously a natural way to
include prior information about parameters within this framework. We give
examples of how to use this hierarchy to compute MAP estimates for linear and
logistic regression as well as sparse precision-matrix estimates in Gaussian
graphical models. In addition, an adaptive group lasso method is derived using
the framework.Comment: Submitted for publication; corrected typo
Harmony and Technology Enhanced Learning
New technologies offer rich opportunities to support education in harmony. In this chapter we consider theoretical perspectives and underlying principles behind technologies for learning and teaching harmony. Such perspectives help in matching existing and future technologies to educational purposes, and to inspire the creative re-appropriation of technologies
- …