4,549 research outputs found
Efficient First Order Methods for Linear Composite Regularizers
A wide class of regularization problems in machine learning and statistics
employ a regularization term which is obtained by composing a simple convex
function \omega with a linear transformation. This setting includes Group Lasso
methods, the Fused Lasso and other total variation methods, multi-task learning
methods and many more. In this paper, we present a general approach for
computing the proximity operator of this class of regularizers, under the
assumption that the proximity operator of the function \omega is known in
advance. Our approach builds on a recent line of research on optimal first
order optimization methods and uses fixed point iterations for numerically
computing the proximity operator. It is more general than current approaches
and, as we show with numerical simulations, computationally more efficient than
available first order methods which do not achieve the optimal rate. In
particular, our method outperforms state of the art O(1/T) methods for
overlapping Group Lasso and matches optimal O(1/T^2) methods for the Fused
Lasso and tree structured Group Lasso.Comment: 19 pages, 8 figure
Geometric Ergodicity of Gibbs Samplers in Bayesian Penalized Regression Models
We consider three Bayesian penalized regression models and show that the
respective deterministic scan Gibbs samplers are geometrically ergodic
regardless of the dimension of the regression problem. We prove geometric
ergodicity of the Gibbs samplers for the Bayesian fused lasso, the Bayesian
group lasso, and the Bayesian sparse group lasso. Geometric ergodicity along
with a moment condition results in the existence of a Markov chain central
limit theorem for Monte Carlo averages and ensures reliable output analysis.
Our results of geometric ergodicity allow us to also provide default starting
values for the Gibbs samplers
The group fused Lasso for multiple change-point detection
We present the group fused Lasso for detection of multiple change-points
shared by a set of co-occurring one-dimensional signals. Change-points are
detected by approximating the original signals with a constraint on the
multidimensional total variation, leading to piecewise-constant approximations.
Fast algorithms are proposed to solve the resulting optimization problems,
either exactly or approximately. Conditions are given for consistency of both
algorithms as the number of signals increases, and empirical evidence is
provided to support the results on simulated and array comparative genomic
hybridization data
Structured sparse CCA for brain imaging genetics via graph OSCAR
Recently, structured sparse canonical correlation analysis (SCCA) has received increased attention in brain imaging genetics studies. It can identify bi-multivariate imaging genetic associations as well as select relevant features with desired structure information. These SCCA methods either use the fused lasso regularizer to induce the smoothness between ordered features, or use the signed pairwise difference which is dependent on the estimated sign of sample correlation. Besides, several other structured SCCA models use the group lasso or graph fused lasso to encourage group structure, but they require the structure/group information provided in advance which sometimes is not available
Clustering in linear mixed models with a group fused lasso penalty
A method is proposed that aims at identifying clusters of individuals that show similar patterns when observed repeatedly. We consider linear mixed models which are widely used for the modeling of longitudinal data. In contrast to the classical assumption of a normal distribution for the random effects a finite mixture of normal distributions is assumed. Typically, the number of mixture components is unknown and has to be chosen, ideally by data driven tools. For this purpose an EM algorithm-based approach is considered that uses a penalized normal mixture as random effects distribution. The penalty term shrinks the pairwise distances of cluster centers based on the group lasso and the fused lasso method. The effect is that individuals with similar time trends are merged into the same cluster. The strength of
regularization is determined by one penalization parameter. For finding the optimal penalization parameter a new model choice criterion is proposed
Efficient First Order Methods for Linear Composite Regularizers
A wide class of regularization problems in machine learning and statistics employ a regularization term which is obtained by composing a simple convex function omega with a linear transformation. This setting includes Group Lasso methods, the Fused Lasso and other total variation methods, multi-task learning methods and many more. In this paper, we present a general approach for computing the proximity operator of this class of regularizers, under the assumption that the proximity operator of the function \omega is known in advance. Our approach builds on a recent line of research on optimal first order optimization methods and uses fixed point iterations for numerically computing the proximity operator. It is more general than current approaches and, as we show with numerical simulations, computationally more efficient than available first order methods which do not achieve the optimal rate. In particular, our method outperforms state of the art O(1/T) methods for overlapping Group Lasso and matches optimal O(1/T2) methods for the Fused Lasso and tree structured Group Lasso
Adaptive Fused LASSO in Grouped Quantile Regression
This paper considers quantile model with grouped explanatory variables. In
order to have the sparsity of the parameter groups but also the sparsity
between two successive groups of variables, we propose and study an adaptive
fused group LASSO quantile estimator. The number of variable groups can be
fixed or divergent. We find the convergence rate under classical assumptions
and we show that the proposed estimator satisfies the oracle properties
- …