Quantile regression in high-dimension with breaking

The paper considers a linear regression model in high-dimension for which the predictive variables can change the influence on the response variable at unknown times (called change-points). Moreover, the particular case of the heavy-tailed errors is considered. In this case, least square method with LASSO or adaptive LASSO penalty can not be used since the theoretical assumptions do not occur or the estimators are not robust. Then, the quantile model with SCAD penalty or median regression with LASSO-type penalty allows, in the same time, to estimate the parameters on every segment and eliminate the irrelevant variables. We show that, for the two penalized estimation methods, the oracle properties is not affected by the change-point estimation. Convergence rates of the estimators for the change-points and for the regression parameters, by the two methods are found. Monte-Carlo simulations illustrate the performance of the methods

Symmetry, regression design, and sampling distributions

When values of regressors are symmetrically disposed, many M-estimators in a wide class of models have a reflection property, namely, that as the signs of the coefficients on regressors are reversed, their estimators' sampling distribution is reflected about the origin. When the coefficients are zero, sign reversal can have no effect. So in this case, the sampling distribution of regression coefficient estimators is symmetric about zero, the estimators are median unbiased and, when moments exist, the estimators are exactly uncorrelated with estimators of other parameters. The result is unusual in that it does not require response variates to have symmetric conditional distributions. It demonstrates the potential importance of covariate design in determining the distributions of estimators, and it is useful in designing and interpreting Monte Carlo experiments. The result is illustrated by a Monte Carlo experiment in which maximum likelihood and symmetrically censored least-squares estimators are calculated for small samples from a censored normal linear regression, Tobit, model. © 1994, Cambridge University Press. All rights reserved

Least absolute deviation estimation of linear econometric models: A literature review

Econometricians generally take for granted that the error terms in the econometric models are generated by distributions having a finite variance. However, since the time of Pareto the existence of error distributions with infinite variance is known. Works of many econometricians, namely, Meyer & Glauber (1964), Fama (1965) and Mandlebroth (1967), on economic data series like prices in financial and commodity markets confirm that infinite variance distributions exist abundantly. The distribution of firms by size, behaviour of speculative prices and various other recent economic phenomena also display similar trends. Further, econometricians generally assume that the disturbance term, which is an influence of innumerably many factors not accounted for in the model, approaches normality according to the Central Limit Theorem. But Bartels (1977) is of the opinion that there are limit theorems, which are just likely to be relevant when considering the sum of number of components in a regression disturbance that leads to non-normal stable distribution characterized by infinite variance. Thus, the possibility of the error term following a non-normal distribution exists. The Least Squares method of estimation of parameters of linear (regression) models performs well provided that the residuals (disturbances or errors) are well behaved (preferably normally or near-normally distributed and not infested with large size outliers) and follow Gauss-Markov assumptions. However, models with the disturbances that are prominently non-normally distributed and contain sizeable outliers fail estimation by the Least Squares method. An intensive research has established that in such cases estimation by the Least Absolute Deviation (LAD) method performs well. This paper is an attempt to survey the literature on LAD estimation of single as well as multi-equation linear econometric models.Lad estimator; Least absolute deviation estimation; econometric model; LAD Estimator; Minimum Absolute Deviation; Robust; Outliers; L1 Estimator; Review of literature

Adaptive confidence intervals for regression functions under shape constraints

Adaptive confidence intervals for regression functions are constructed under shape constraints of monotonicity and convexity. A natural benchmark is established for the minimum expected length of confidence intervals at a given function in terms of an analytic quantity, the local modulus of continuity. This bound depends not only on the function but also the assumed function class. These benchmarks show that the constructed confidence intervals have near minimum expected length for each individual function, while maintaining a given coverage probability for functions within the class. Such adaptivity is much stronger than adaptive minimaxity over a collection of large parameter spaces.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1068 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Statistical and Computational Tradeoff in Genetic Algorithm-Based Estimation

When a Genetic Algorithm (GA), or a stochastic algorithm in general, is employed in a statistical problem, the obtained result is affected by both variability due to sampling, that refers to the fact that only a sample is observed, and variability due to the stochastic elements of the algorithm. This topic can be easily set in a framework of statistical and computational tradeoff question, crucial in recent problems, for which statisticians must carefully set statistical and computational part of the analysis, taking account of some resource or time constraints. In the present work we analyze estimation problems tackled by GAs, for which variability of estimates can be decomposed in the two sources of variability, considering some constraints in the form of cost functions, related to both data acquisition and runtime of the algorithm. Simulation studies will be presented to discuss the statistical and computational tradeoff question.Comment: 17 pages, 5 figure

Estimating long range dependence: finite sample properties and confidence intervals

A major issue in financial economics is the behavior of asset returns over long horizons. Various estimators of long range dependence have been proposed. Even though some have known asymptotic properties, it is important to test their accuracy by using simulated series of different lengths. We test R/S analysis, Detrended Fluctuation Analysis and periodogram regression methods on samples drawn from Gaussian white noise. The DFA statistics turns out to be the unanimous winner. Unfortunately, no asymptotic distribution theory has been derived for this statistics so far. We were able, however, to construct empirical (i.e. approximate) confidence intervals for all three methods. The obtained values differ largely from heuristic values proposed by some authors for the R/S statistics and are very close to asymptotic values for the periodogram regression method.Comment: 16 pages, 11 figures New version: 14 pages (smaller fonts), 11 figures, new Section on application

Confidence Sets Based on Penalized Maximum Likelihood Estimators in Gaussian Regression

Confidence intervals based on penalized maximum likelihood estimators such as the LASSO, adaptive LASSO, and hard-thresholding are analyzed. In the known-variance case, the finite-sample coverage properties of such intervals are determined and it is shown that symmetric intervals are the shortest. The length of the shortest intervals based on the hard-thresholding estimator is larger than the length of the shortest interval based on the adaptive LASSO, which is larger than the length of the shortest interval based on the LASSO, which in turn is larger than the standard interval based on the maximum likelihood estimator. In the case where the penalized estimators are tuned to possess the `sparsity property', the intervals based on these estimators are larger than the standard interval by an order of magnitude. Furthermore, a simple asymptotic confidence interval construction in the `sparse' case, that also applies to the smoothly clipped absolute deviation estimator, is discussed. The results for the known-variance case are shown to carry over to the unknown-variance case in an appropriate asymptotic sense.Comment: second revision: new title, some comments added, proofs moved to appendi

Linearization of randomly weighted empiricals under long range dependence with application to nonlinear regression quantiles

This paper discusses some asymptotic uniform linearity results of randomly weighted empirical processes based on long range dependent random variables+ These results are subsequently used to linearize nonlinear regression quantiles in a nonlinear regression model with long range dependent errors, where the design variables can be either random or nonrandom+ These, in turn, yield the limiting behavior of the nonlinear regression quantiles+ As a corollary, we obtain the limiting behavior of the least absolute deviation estimator and the trimmed mean estimator of the parameters of the nonlinear regression model+ Some of the limiting properties are in striking contrast with the corresponding properties of a nonlinear regression model under independent and identically distributed error random variables+ The paper also discusses an extension of rank score statistic in a nonlinear regression model