Regularization Methods for Fitting Linear Models with Small Sample Sizes: Fitting the Lasso Estimator using R

Researchers and data analysts are sometimes faced with the problem of very small samples, where the number of variables approaches or exceeds the overall sample size; i.e. high dimensional data. In such cases, standard statistical models such as regression or analysis of variance cannot be used, either because the resulting parameter estimates exhibit very high variance and can therefore not be trusted, or because the statistical algorithm cannot converge on parameter estimates at all. There exist an alternative set of model estimation procedures, known collectively as regularization methods, which can be used in such circumstances, and which have been shown through simulation research to yield accurate parameter estimates. The purpose of this paper is to describe, for those unfamiliar with them, the most popular of these regularization methods, the lasso, and to demonstrate its use on an actual high dimensional dataset involving adults with autism, using the R software language. Results of analyses involving relating measures of executive functioning with a full scale intelligence test score are presented, and implications of using these models are discussed. Accessed 4,969 times on https://pareonline.net from May 08, 2016 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right

Multi-Period Trading via Convex Optimization

We consider a basic model of multi-period trading, which can be used to evaluate the performance of a trading strategy. We describe a framework for single-period optimization, where the trades in each period are found by solving a convex optimization problem that trades off expected return, risk, transaction cost and holding cost such as the borrowing cost for shorting assets. We then describe a multi-period version of the trading method, where optimization is used to plan a sequence of trades, with only the first one executed, using estimates of future quantities that are unknown when the trades are chosen. The single-period method traces back to Markowitz; the multi-period methods trace back to model predictive control. Our contribution is to describe the single-period and multi-period methods in one simple framework, giving a clear description of the development and the approximations made. In this paper we do not address a critical component in a trading algorithm, the predictions or forecasts of future quantities. The methods we describe in this paper can be thought of as good ways to exploit predictions, no matter how they are made. We have also developed a companion open-source software library that implements many of the ideas and methods described in the paper

Constructing Optimal Sparse Portfolios Using Regularization Methods

The ideas of Markowitz indisputably constitute a milestone in portfolio theory, even though the resulting mean-variance portfolios typically exhibit an unsatisfying out-of-sample performance, especially when the number of securities is large and that of observations is not. The bad performance is caused by estimation errors in the covariance matrix and in the expected return vector that can deposit unhindered in the portfolio weights. Recent studies show that imposing a penalty in form of a l1-norm of the asset weights regularizes the problem, thereby improving the out-of-sample performance of the optimized portfolios. Simultaneously, l1-regularization selects a subset of assets to invest in from a pool of candidates that is often very large. However, l1-regularization might lead to the construction of biased solutions. We propose to tackle this issue by considering several alternative penalties proposed in non-financial contexts. Moreover we propose a simple new type of penalty that explicitly considers funancial information. We show empirically that these alternative penalties can lead to the construction of portfolios with superior out-of-sample performance in comparison to the state-of-the art l1-regularized portfolios and several standard benchmarks, especially in high dimensional problems. The empirical analysis is conducted with various U.S.-stock market datasets

Constructing optimal sparse portfolios using regularization methods

The ideas of Markowitz indisputably constitute a milestone in portfolio theory, even though the resulting mean-variance portfolios typically exhibit an unsatisfying out-of-sample performance, especially when the number of securities is large and that of observations is not. The bad performance is caused by estimation errors in the covariance matrix and in the expected return vector that can deposit unhindered in the portfolio weights. Recent studies show that imposing a penalty in form of a l1-norm of the asset weights regularizes the problem, thereby improving the out-of-sample performance of the optimized portfolios. Simultaneously, l1-regularization selects a subset of assets to invest in from a pool of candidates that is often very large. However, l1-regularization might lead to the construction of biased solutions. We propose to tackle this issue by considering several alternative penalties proposed in non-financial contexts. Moreover we propose a simple new type of penalty that explicitly considers financial information. We show empirically that these alternative penalties can lead to the construction of portfolios with superior out-of-sample performance in comparison to the state-of-the-art l1-regularized portfolios and several standard benchmarks, especially in high dimensional problems. The empirical analysis is conducted with various U.S.-stock market datasets

Quantile regression methods in economics and finance

In the recent years, quantile regression methods have attracted relevant interest in the statistical and econometric literature. This phenomenon is due to the advantages arising from the quantile regression approach, mainly the robustness of the results and the possibility to analyse several quantiles of a given random variable. Such as features are particularly appealing in the context of economic and financial data, where extreme events assume critical importance. The present thesis is based on quantile regression, with focus on the economic and financial environment. First of all, we propose new approaches in developing asset allocation strategies on the basis of quantile regression and regularization techniques. It is well known that quantile regression model minimizes the portfolio extreme risk, whenever the attention is placed on the estimation of the response variable left quantiles. We show that, by considering the entire conditional distribution of the dependent variable, it is possible to optimize different risk and performance indicators. In particular, we introduce a risk-adjusted profitability measure, useful in evaluating financial portfolios under a pessimistic perspective, since the reward contribution is net of the most favorable outcomes. Moreover, as we consider large portfolios, we also cope with the dimensionality issue by introducing an l1-norm penalty on the assets weights. Secondly, we focus on the determinants of equity risk and their forecasting implications. Several market and macro-level variables influence the evolution of equity risk in addition to the well-known volatility persistence. However, the impact of those covariates might change depending on the risk level, being different between low and high volatility states. By combining equity risk estimates, obtained from the Realized Range Volatility, corrected for microstructure noise and jumps, and quantile regression methods, we evaluate, in a forecasting perspective, the impact of the equity risk determinants in different volatility states and, without distributional assumptions on the realized range innovations, we recover both the points and the conditional distribution forecasts. In addition, we analyse how the relationships among the involved variables evolve over time, through a rolling window procedure. The results show evidence of the selected variables' relevant impacts and, particularly during periods of market stress, highlight heterogeneous effects across quantiles. Finally, we study the dynamic impact of uncertainty in causing and forecasting the distribution of oil returns and risk. We analyse the relevance of recently developed news-based measures of economic policy uncertainty and equity market uncertainty in causing and predicting the conditional quantiles and distribution of the crude oil variations, defined both as returns and squared returns. For this purpose, on the one hand, we study the causality relations in quantiles through a non-parametric testing method; on the other hand, we forecast the conditional distribution on the basis of the quantile regression approach and the predictive accuracy is evaluated by means of several suitable tests. Given the presence of structural breaks over time, we implement a rolling window procedure to capture the dynamic relations among the variables