8,379 research outputs found
Elastic-Net Regularization in Learning Theory
Within the framework of statistical learning theory we analyze in detail the
so-called elastic-net regularization scheme proposed by Zou and Hastie for the
selection of groups of correlated variables. To investigate on the statistical
properties of this scheme and in particular on its consistency properties, we
set up a suitable mathematical framework. Our setting is random-design
regression where we allow the response variable to be vector-valued and we
consider prediction functions which are linear combination of elements ({\em
features}) in an infinite-dimensional dictionary. Under the assumption that the
regression function admits a sparse representation on the dictionary, we prove
that there exists a particular ``{\em elastic-net representation}'' of the
regression function such that, if the number of data increases, the elastic-net
estimator is consistent not only for prediction but also for variable/feature
selection. Our results include finite-sample bounds and an adaptive scheme to
select the regularization parameter. Moreover, using convex analysis tools, we
derive an iterative thresholding algorithm for computing the elastic-net
solution which is different from the optimization procedure originally proposed
by Zou and HastieComment: 32 pages, 3 figure
On an unsupervised method for parameter selection for the elastic net
Despite recent advances in regularization theory, the issue of parameter selection still remains a challenge for most applications. In a recent work the framework of statistical learning was used to approximate the optimal Tikhonov regularization parameter from noisy data. In this work, we improve their results and extend the analysis to the elastic net regularization. Furthermore, we design a data-driven, automated algorithm for the computation of an approximate regularization parameter. Our analysis combines statistical learning theory with insights from regularization theory. We compare our approach with state-of-the-art parameter selection criteria and show that it has superior accuracy
Fast learning rate of multiple kernel learning: Trade-off between sparsity and smoothness
We investigate the learning rate of multiple kernel learning (MKL) with
and elastic-net regularizations. The elastic-net regularization is a
composition of an -regularizer for inducing the sparsity and an
-regularizer for controlling the smoothness. We focus on a sparse
setting where the total number of kernels is large, but the number of nonzero
components of the ground truth is relatively small, and show sharper
convergence rates than the learning rates have ever shown for both and
elastic-net regularizations. Our analysis reveals some relations between the
choice of a regularization function and the performance. If the ground truth is
smooth, we show a faster convergence rate for the elastic-net regularization
with less conditions than -regularization; otherwise, a faster
convergence rate for the -regularization is shown.Comment: Published in at http://dx.doi.org/10.1214/13-AOS1095 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org). arXiv admin note: text overlap with
arXiv:1103.043
Pathway-Based Genomics Prediction using Generalized Elastic Net.
We present a novel regularization scheme called The Generalized Elastic Net (GELnet) that incorporates gene pathway information into feature selection. The proposed formulation is applicable to a wide variety of problems in which the interpretation of predictive features using known molecular interactions is desired. The method naturally steers solutions toward sets of mechanistically interlinked genes. Using experiments on synthetic data, we demonstrate that pathway-guided results maintain, and often improve, the accuracy of predictors even in cases where the full gene network is unknown. We apply the method to predict the drug response of breast cancer cell lines. GELnet is able to reveal genetic determinants of sensitivity and resistance for several compounds. In particular, for an EGFR/HER2 inhibitor, it finds a possible trans-differentiation resistance mechanism missed by the corresponding pathway agnostic approach
- …