Adaptive Robust Methodology for Parameter Estimation and Variable Selection

The dissertation consists of three distinct but related projects. We consider regression model fitting, variable selection in regression, and autocorrelation estimation in time series. In each procedure we formulate the problem in terms of minimizing an objective function which adapts to the given data. First we propose a robust M-estimation procedure for regression. The main purpose of the proposed methodology is to develop a procedure that adapts to light/heavy tailed, symmetric/asymmetric distributions with/without outliers. We focus on studying the properties of the maximum likelihood estimator of the asymmetric exponential power distribution, a broad distribution class that holds both Normal and asymmetric Laplace distributions as special cases. The proposed methodology unifies least squares and quantile regression in a data driven manner to capture both tail decay and asymmetry of the underlying distributions. Finite sample performance of the method is exhibited via extensive Monte Carlo simulation and real data applications. Second, we capitalize on the success of the proposed method and extend it to a variable selection procedure that selects the important predictors under a sparse setting. Quantile regression Lasso, i.e., quantile regession with $L_1$ norm on the regression coefficients for regularization is a robust technique to perform variable selection. However which quantile should be adopted is unclear. The proposed methodology introduces a way to choose the most ``informative\u27 quantile of interest that is used in the adaptive quantile regression Lasso. A modified BIC criterion is used to select the optimal tuning parameter. The proposed procedure selects the quantile based on the log-likelihood of the asymmetric Laplace distribution, and aims to perform the best quantile regression Lasso which is confirmed in both simulation study and a real data analysis. Third, we focus on alleviating the underestimation issue of the sample autocorrelation in linear stationary time series. We first formulate autocorrelation estimation into a least squares problem and then apply a penalization to regulate the autocorrelation estimate. An adaptive sequence is proposed for tuning parameter and is shown to work well for stationary time series when the sample size is small and correlation is high

Prior elicitation and variable selection for bayesian quantile regression

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Bayesian subset selection suffers from three important difficulties: assigning priors over model space, assigning priors to all components of the regression coefficients vector given a specific model and Bayesian computational efficiency (Chen et al., 1999). These difficulties become more challenging in Bayesian quantile regression framework when one is interested in assigning priors that depend on different quantile levels. The objective of Bayesian quantile regression (BQR), which is a newly proposed tool, is to deal with unknown parameters and model uncertainty in quantile regression (QR). However, Bayesian subset selection in quantile regression models is usually a difficult issue due to the computational challenges and nonavailability of conjugate prior distributions that are dependent on the quantile level. These challenges are rarely addressed via either penalised likelihood function or stochastic search variable selection (SSVS). These methods typically use symmetric prior distributions for regression coefficients, such as the Gaussian and Laplace, which may be suitable for median regression. However, an extreme quantile regression should have different regression coefficients from the median regression, and thus the priors for quantile regression coefficients should depend on quantiles. This thesis focuses on three challenges: assigning standard quantile dependent prior distributions for the regression coefficients, assigning suitable quantile dependent priors over model space and achieving computational efficiency. The first of these challenges is studied in Chapter 2 in which a quantile dependent prior elicitation scheme is developed. In particular, an extension of the Zellners prior which allows for a conditional conjugate prior and quantile dependent prior on Bayesian quantile regression is proposed. The prior is generalised in Chapter 3 by introducing a ridge parameter to address important challenges that may arise in some applications, such as multicollinearity and overfitting problems. The proposed prior is also used in Chapter 4 for subset selection of the fixed and random coefficients in a linear mixedeffects QR model. In Chapter 5 we specify normal-exponential prior distributions for the regression coefficients which can provide adaptive shrinkage and represent an alternative model to the Bayesian Lasso quantile regression model. For the second challenge, we assign a quantile dependent prior over model space in Chapter 2. The prior is based on the percentage bend correlation which depends on the quantile level. This prior is novel and is used in Bayesian regression for the first time. For the third challenge of computational efficiency, Gibbs samplers are derived and setup to facilitate the computation of the proposed methods. In addition to the three major aforementioned challenges this thesis also addresses other important issues such as the regularisation in quantile regression and selecting both random and fixed effects in mixed quantile regression models

Bayesian adaptive lasso quantile regression

Recently, variable selection by penalized likelihood has attracted much research interest. In this paper, we propose adaptive Lasso quantile regression (BALQR) from a Bayesian perspective. The method extends the Bayesian Lasso quantile regression by allowing different penalization parameters for different regression coefficients. Inverse gamma prior distributions are placed on the penalty parameters. We treat the hyperparameters of the inverse gamma prior as unknowns and estimate them along with the other parameters. A Gibbs sampler is developed to simulate the parameters from the posterior distributions. Through simulation studies and analysis of a prostate cancer dataset, we compare the performance of the BALQR method proposed with six existing Bayesian and non-Bayesian methods. The simulation studies and the prostate cancer data analysis indicate that the BALQR method performs well in comparison to the other approaches

Quantile regression in high-dimension with breaking

The paper considers a linear regression model in high-dimension for which the predictive variables can change the influence on the response variable at unknown times (called change-points). Moreover, the particular case of the heavy-tailed errors is considered. In this case, least square method with LASSO or adaptive LASSO penalty can not be used since the theoretical assumptions do not occur or the estimators are not robust. Then, the quantile model with SCAD penalty or median regression with LASSO-type penalty allows, in the same time, to estimate the parameters on every segment and eliminate the irrelevant variables. We show that, for the two penalized estimation methods, the oracle properties is not affected by the change-point estimation. Convergence rates of the estimators for the change-points and for the regression parameters, by the two methods are found. Monte-Carlo simulations illustrate the performance of the methods

Choosing the Right Spatial Weighting Matrix in a Quantile Regression Model

This paper proposes computationally tractable methods for selecting the appropriate spatial weighting matrix in the context of a spatial quantile regression model. This selection is a notoriously difficult problem even in linear spatial models and is even more difficult in a quantile regression setup. The proposal is illustrated by an empirical example and manages to produce tractable models. One important feature of the proposed methodology is that by allowing different degrees and forms of spatial dependence across quantiles it further relaxes the usual quantile restriction attributable to the linear quantile regression. In this way we can obtain a more robust, with regard to potential functional misspecification, model, but nevertheless preserve the parametric rate of convergence and the established inferential apparatus associated with the linear quantile regression approach

Predictive densities for day-ahead electricity prices using time-adaptive quantile regression

A large part of the decision-making problems actors of the power system are facing on a daily basis requires scenarios for day-ahead electricity market prices. These scenarios are most likely to be generated based on marginal predictive densities for such prices, then enhanced with a temporal dependence structure. A semi-parametric methodology for generating such densities is presented: it includes: (i) a time-adaptive quantile regression model for the 5%–95% quantiles; and (ii) a description of the distribution tails with exponential distributions. The forecasting skill of the proposed model is compared to that of four benchmark approaches and the well-known the generalist autoregressive conditional heteroskedasticity (GARCH) model over a three-year evaluation period. While all benchmarks are outperformed in terms of forecasting skill overall, the superiority of the semi-parametric model over the GARCH model lies in the former’s ability to generate reliable quantile estimates

A Quantile Regression Analysis of the Effect of Farmers’ Attitudes and Perceptions on Market Participation

The objective of this study is to investigate the subjective determinants of farmers’ participation in output markets in five EU New Member States (NMS) characterised by large semi-subsistence sectors. It employs quantile regression to model market participation reflecting the heterogeneity amongst farmers. The study also uses the Bayesian adaptive lasso to simultaneously select important covariates and estimate the corresponding quantile regression models. The empirical results show that only two variables affect all quantiles, while their effect varies across quantiles. Some of the remaining variables affect the share of output sold at the lower quantiles (i.e. for subsistence- and semi-subsistence-oriented farmers) only, whereas other variables are only significant at the upper quantiles (i.e. for more commercially oriented farms). Advisory services, and particularly agricultural business advice, and information and advice on markets and prices can facilitate the market participation of subsistence-oriented farms

Local Quantile Regression

Quantile regression is a technique to estimate conditional quantile curves. It provides a comprehensive picture of a response contingent on explanatory variables. In a flexible modeling framework, a specific form of the conditional quantile curve is not a priori fixed. % Indeed, the majority of applications do not per se require specific functional forms. This motivates a local parametric rather than a global fixed model fitting approach. A nonparametric smoothing estimator of the conditional quantile curve requires to balance between local curvature and stochastic variability. In this paper, we suggest a local model selection technique that provides an adaptive estimator of the conditional quantile regression curve at each design point. Theoretical results claim that the proposed adaptive procedure performs as good as an oracle which would minimize the local estimation risk for the problem at hand. We illustrate the performance of the procedure by an extensive simulation study and consider a couple of applications: to tail dependence analysis for the Hong Kong stock market and to analysis of the distributions of the risk factors of temperature dynamics