1,922 research outputs found
Forecasting VARMA processes using VAR models and subspace-based state space models
VAR modelling is a frequent technique in econometrics for linear processes. VAR modelling offers some desirable features such as relatively simple procedures for model specification (order selection) and the possibility of obtaining quick non-iterative maximum likelihood estimates of the system parameters. However, if the process under study follows a finite-order VARMA structure, it cannot be equivalently represented by any finite-order VAR model. On the other hand, a finite-order state space model can represent a finite-order VARMA process exactly, and, for state-space modelling, subspace algorithms allow for quick and non-iterative estimates of the system parameters, as well as for simple specification procedures. Given the previous facts, we check in this paper whether subspace-based state space models provide better forecasts than VAR models when working with VARMA data generating processes. In a simulation study we generate samples from different VARMA data generating processes, obtain VAR-based and state-space-based models for each generating process and compare the predictive power of the obtained models. Different specification and estimation algorithms are considered; in particular, within the subspace family, the CCA (Canonical Correlation Analysis) algorithm is the selected option to obtain state-space models. Our results indicate that when the MA parameter of an ARMA process is close to 1, the CCA state space models are likely to provide better forecasts than the AR models. We also conduct a practical comparison (for two cointegrated economic time series) of the predictive power of Johansen restricted-VAR (VEC) models with the predictive power of state space models obtained by the CCA subspace algorithm, including a density forecasting analysis.subspace algorithms; VAR; forecasting; cointegration; Johansen; CCA
Fisher Lecture: Dimension Reduction in Regression
Beginning with a discussion of R. A. Fisher's early written remarks that
relate to dimension reduction, this article revisits principal components as a
reductive method in regression, develops several model-based extensions and
ends with descriptions of general approaches to model-based and model-free
dimension reduction in regression. It is argued that the role for principal
components and related methodology may be broader than previously seen and that
the common practice of conditioning on observed values of the predictors may
unnecessarily limit the choice of regression methodology.Comment: This paper commented in: [arXiv:0708.3776], [arXiv:0708.3777],
[arXiv:0708.3779]. Rejoinder in [arXiv:0708.3781]. Published at
http://dx.doi.org/10.1214/088342306000000682 in the Statistical Science
(http://www.imstat.org/sts/) by the Institute of Mathematical Statistics
(http://www.imstat.org
Cointegration Analysis with State Space Models
This paper presents and exemplifies results developed for cointegration analysis with state space models by Bauer and Wagner in a series of papers. Unit root processes, cointegration and polynomial cointegration are defined. Based upon these definitions the major part of the paper discusses how state space models, which are equivalent to VARMA models, can be fruitfully employed for cointegration analysis. By means of detailing the cases most relevant for empirical applications, the I(1), MFI(1) and I(2) cases, a canonical representation is developed and thereafter some available statistical results are briefly mentioned.State space models, unit roots, cointegration, polynomial cointegration, pseudo maximum likelihood estimation, subspace algorithms
Testing predictor contributions in sufficient dimension reduction
We develop tests of the hypothesis of no effect for selected predictors in
regression, without assuming a model for the conditional distribution of the
response given the predictors. Predictor effects need not be limited to the
mean function and smoothing is not required. The general approach is based on
sufficient dimension reduction, the idea being to replace the predictor vector
with a lower-dimensional version without loss of information on the regression.
Methodology using sliced inverse regression is developed in detail
Sparse canonical correlation analysis from a predictive point of view
Canonical correlation analysis (CCA) describes the associations between two
sets of variables by maximizing the correlation between linear combinations of
the variables in each data set. However, in high-dimensional settings where the
number of variables exceeds the sample size or when the variables are highly
correlated, traditional CCA is no longer appropriate. This paper proposes a
method for sparse CCA. Sparse estimation produces linear combinations of only a
subset of variables from each data set, thereby increasing the interpretability
of the canonical variates. We consider the CCA problem from a predictive point
of view and recast it into a regression framework. By combining an alternating
regression approach together with a lasso penalty, we induce sparsity in the
canonical vectors. We compare the performance with other sparse CCA techniques
in different simulation settings and illustrate its usefulness on a genomic
data set
Learning Mixtures of Linear Classifiers
We consider a discriminative learning (regression) problem, whereby the
regression function is a convex combination of k linear classifiers. Existing
approaches are based on the EM algorithm, or similar techniques, without
provable guarantees. We develop a simple method based on spectral techniques
and a `mirroring' trick, that discovers the subspace spanned by the
classifiers' parameter vectors. Under a probabilistic assumption on the feature
vector distribution, we prove that this approach has nearly optimal statistical
efficiency
Adaptive Reduced Rank Regression
We study the low rank regression problem , where and are and dimensional
vectors respectively. We consider the extreme high-dimensional setting where
the number of observations is less than . Existing algorithms
are designed for settings where is typically as large as
. This work provides an efficient algorithm which
only involves two SVD, and establishes statistical guarantees on its
performance. The algorithm decouples the problem by first estimating the
precision matrix of the features, and then solving the matrix denoising
problem. To complement the upper bound, we introduce new techniques for
establishing lower bounds on the performance of any algorithm for this problem.
Our preliminary experiments confirm that our algorithm often out-performs
existing baselines, and is always at least competitive.Comment: 40 page
Calibrated Multivariate Regression with Application to Neural Semantic Basis Discovery
We propose a calibrated multivariate regression method named CMR for fitting
high dimensional multivariate regression models. Compared with existing
methods, CMR calibrates regularization for each regression task with respect to
its noise level so that it simultaneously attains improved finite-sample
performance and tuning insensitiveness. Theoretically, we provide sufficient
conditions under which CMR achieves the optimal rate of convergence in
parameter estimation. Computationally, we propose an efficient smoothed
proximal gradient algorithm with a worst-case numerical rate of convergence
\cO(1/\epsilon), where is a pre-specified accuracy of the
objective function value. We conduct thorough numerical simulations to
illustrate that CMR consistently outperforms other high dimensional
multivariate regression methods. We also apply CMR to solve a brain activity
prediction problem and find that it is as competitive as a handcrafted model
created by human experts. The R package \texttt{camel} implementing the
proposed method is available on the Comprehensive R Archive Network
\url{http://cran.r-project.org/web/packages/camel/}.Comment: Journal of Machine Learning Research, 201
- …