1,515 research outputs found
A Dantzig Selector Approach to Temporal Difference Learning
LSTD is a popular algorithm for value function approximation. Whenever the
number of features is larger than the number of samples, it must be paired with
some form of regularization. In particular, L1-regularization methods tend to
perform feature selection by promoting sparsity, and thus, are well-suited for
high-dimensional problems. However, since LSTD is not a simple regression
algorithm, but it solves a fixed--point problem, its integration with
L1-regularization is not straightforward and might come with some drawbacks
(e.g., the P-matrix assumption for LASSO-TD). In this paper, we introduce a
novel algorithm obtained by integrating LSTD with the Dantzig Selector. We
investigate the performance of the proposed algorithm and its relationship with
the existing regularized approaches, and show how it addresses some of their
drawbacks.Comment: Appears in Proceedings of the 29th International Conference on
Machine Learning (ICML 2012
A machine learning approach to portfolio pricing and risk management for high-dimensional problems
We present a general framework for portfolio risk management in discrete
time, based on a replicating martingale. This martingale is learned from a
finite sample in a supervised setting. The model learns the features necessary
for an effective low-dimensional representation, overcoming the curse of
dimensionality common to function approximation in high-dimensional spaces. We
show results based on polynomial and neural network bases. Both offer superior
results to naive Monte Carlo methods and other existing methods like
least-squares Monte Carlo and replicating portfolios.Comment: 30 pages (main), 10 pages (appendix), 3 figures, 22 table
SCAD-penalized regression in high-dimensional partially linear models
We consider the problem of simultaneous variable selection and estimation in
partially linear models with a divergent number of covariates in the linear
part, under the assumption that the vector of regression coefficients is
sparse. We apply the SCAD penalty to achieve sparsity in the linear part and
use polynomial splines to estimate the nonparametric component. Under
reasonable conditions, it is shown that consistency in terms of variable
selection and estimation can be achieved simultaneously for the linear and
nonparametric components. Furthermore, the SCAD-penalized estimators of the
nonzero coefficients are shown to have the asymptotic oracle property, in the
sense that it is asymptotically normal with the same means and covariances that
they would have if the zero coefficients were known in advance. The finite
sample behavior of the SCAD-penalized estimators is evaluated with simulation
and illustrated with a data set.Comment: Published in at http://dx.doi.org/10.1214/07-AOS580 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
"Pre-conditioning" for feature selection and regression in high-dimensional problems
We consider regression problems where the number of predictors greatly
exceeds the number of observations. We propose a method for variable selection
that first estimates the regression function, yielding a "pre-conditioned"
response variable. The primary method used for this initial regression is
supervised principal components. Then we apply a standard procedure such as
forward stepwise selection or the LASSO to the pre-conditioned response
variable. In a number of simulated and real data examples, this two-step
procedure outperforms forward stepwise selection or the usual LASSO (applied
directly to the raw outcome). We also show that under a certain Gaussian latent
variable model, application of the LASSO to the pre-conditioned response
variable is consistent as the number of predictors and observations increases.
Moreover, when the observational noise is rather large, the suggested procedure
can give a more accurate estimate than LASSO. We illustrate our method on some
real problems, including survival analysis with microarray data
- …