254 research outputs found
Clusterpath An Algorithm for Clustering using Convex Fusion Penalties
International audienceWe present a new clustering algorithm by proposing a convex relaxation of hierarchical clustering, which results in a family of objective functions with a natural geometric interpretation. We give efficient algorithms for calculating the continuous regularization path of solutions, and discuss relative advantages of the parameters. Our method experimentally gives state-of-the-art results similar to spectral clustering for non-convex clusters, and has the added benefit of learning a tree structure from the data
SLOPE - Adaptive variable selection via convex optimization
We introduce a new estimator for the vector of coefficients in the
linear model , where has dimensions with
possibly larger than . SLOPE, short for Sorted L-One Penalized Estimation,
is the solution to where
and are the
decreasing absolute values of the entries of . This is a convex program and
we demonstrate a solution algorithm whose computational complexity is roughly
comparable to that of classical procedures such as the Lasso. Here,
the regularizer is a sorted norm, which penalizes the regression
coefficients according to their rank: the higher the rank - that is, stronger
the signal - the larger the penalty. This is similar to the Benjamini and
Hochberg [J. Roy. Statist. Soc. Ser. B 57 (1995) 289-300] procedure (BH) which
compares more significant -values with more stringent thresholds. One
notable choice of the sequence is given by the BH critical
values , where and
is the quantile of a standard normal distribution. SLOPE aims to
provide finite sample guarantees on the selected model; of special interest is
the false discovery rate (FDR), defined as the expected proportion of
irrelevant regressors among all selected predictors. Under orthogonal designs,
SLOPE with provably controls FDR at level .
Moreover, it also appears to have appreciable inferential properties under more
general designs while having substantial power, as demonstrated in a series
of experiments running on both simulated and real data.Comment: Published at http://dx.doi.org/10.1214/15-AOAS842 in the Annals of
Applied Statistics (http://www.imstat.org/aoas/) by the Institute of
Mathematical Statistics (http://www.imstat.org
Parameter Estimation with the Ordered Regularization via an Alternating Direction Method of Multipliers
Regularization is a popular technique in machine learning for model
estimation and avoiding overfitting. Prior studies have found that modern
ordered regularization can be more effective in handling highly correlated,
high-dimensional data than traditional regularization. The reason stems from
the fact that the ordered regularization can reject irrelevant variables and
yield an accurate estimation of the parameters. How to scale up the ordered
regularization problems when facing the large-scale training data remains an
unanswered question. This paper explores the problem of parameter estimation
with the ordered -regularization via Alternating Direction Method of
Multipliers (ADMM), called ADMM-O. The advantages of ADMM-O
include (i) scaling up the ordered to a large-scale dataset, (ii)
predicting parameters correctly by excluding irrelevant variables
automatically, and (iii) having a fast convergence rate. Experiment results on
both synthetic data and real data indicate that ADMM-O can perform
better than or comparable to several state-of-the-art baselines
- …