Forecasting stock market out-of-sample with regularised regression training techniques

Forecasting stock market out-of-sample is a major concern to researchers in finance and emerging markets. This research focuses mainly on the application of regularised Regression Training (RT) techniques to forecast monthly equity premium out-of-sample recursively with an expanding window method. A broad category of sophisticated regularised RT models involving model complexity were employed. The regularised RT models which include Ridge, Forward-Backward (FOBA) Ridge, Least Absolute Shrinkage and Selection Operator (LASSO), Relaxed LASSO, Elastic Net and Least Angle Regression were trained and used to forecast the equity premium out-of-sample. In this study, the empirical investigation of the Regularised RT models demonstrate significant evidence of equity premium predictability both statistically and economically relative to the benchmark historical average, delivering significant utility gains. Overall, the Ridge gives the best statistical performance evaluation results while the LASSO appeared to be most economical meaningful. They seek to provide meaningful economic information on mean-variance portfolio investment for investors who are timing the market to earn future gains at minimal risk. Thus, the forecasting models appeared to guarantee an investor in a market setting who optimally reallocates a monthly portfolio between equities and risk-free treasury bills using equity premium forecasts at minimal risk

Essays in portfolio optimization

This thesis presents three essays in the topic of portfolio optimization and index tracking. The first essay is a critique of the Tangency Portfolio (TP). The TP has paramount theoretical importance in the Modern Portfolio Theory, however, its performance is far from satisfactory. The TP presents unstable weights, what increases the portfolio turnover and, consequently, its transaction costs. Furthermore, the denominator of the TP is frequently close to zero, what can result in extreme weights, precluding the portfolio from being well diversified. There is also the possibility that the TP’s denominator is negative, causing an inversion of the allocation vector’s signal and the delivery of a poor portfolio. This work compares the TP with other portfolios and finds that the TP always comes at the bottom. The work also offers propositions that show that the TP can be outperformed (in terms of utility) by other portfolios. The second essay provides an explicit derivation to the equivalence between the minimum variance portfolio of excess returns on a benchmark and the minimum Tracking Error volatility portfolio. This result relies on the Sherman-Morrison formula. The essay also presents an equivalence of those results to an OLS regression with constrained beta. Further, the essay uses the first equivalence result to find a tracking portfolio using the approach of Kempf and Memmel (2006). The third essay compares the performance of passive investment portfolio with a small number of assets (sparse index tracking portfolios) using different feature selection algorithms. To that end we provide an empirical examination with two datasets: one American and one Brazilian. To isolate the effect of the selection methods, we separate the asset selection and the asset allocation phase. In the asset allocation phase, we follow Liu (2009), and select minimum Tracking Error volatility portfolios. The selection methods used are the backward stepwise selection, forward stepwise selection, and the lasso. Our results show that, in the Brazilian case, the lasso selection method is the best tracker. In the American case, the lasso presents better risk-adjusted performance, but this is due to higher mean returns, not lower volatility. This is undesirable in our case. One highlight of this essay is that the best tracker for the American Dataset uses the backward iteration algorithm (a simple method that receive little attention in the literature).A presente tese apresenta três ensaios com o tema de otimização de carteiras e index tracking. O primeiro ensaio é uma critica ao Portfolio de Tangência (TP). O TP tem grande importância teórica na Moderna Teoria do Portfólio; porém, o seu desempenho está longe de ser satisfatório. O TP apresenta pesos instáveis, o que aumenta o turnover da carteira e, consequentemente, os seus custos de transação. Além disso, frequentemente, o denominador do TP é perto de zero, o que pode resultar em pesos extremos, o que impede a boa diversificação dos pesos da carteira. Também há a possibilidade de o denominador do TP ser negativo, o que causa a inversão do sinal do vetor de alocação, resultando em um portfólio ruim. Este trabalho compara o TP com outras carteiras e chega a conclusão que o TP tem um desempenho pífio. O trabalho também oferece proposições que mostram que o TP pode ser superado (em termos de utilidade) por outras carteiras. O segundo ensaio oferece uma derivação explícita da equivalência entre a carteira de variância mínima e a carteira de mínima volatilidade do Tracking Error. Esse resultado depende da fórmula de Sherman-Morrison . O ensaio também apresenta uma equivalência desses resultados com uma regressão restringida de Mínimos Quadrados. Além disso, o ensaio usa a primeira equivalência para achar uma carteira de tracking usando a abordagem de Kempf and Memmel (2006). O terceiro ensaio compara a performance de carteiras de investimento passivo com um número pequeno de ativos (carteiras esparsas de index tracking usando diferentes algoritmos de feature selection. Para isso, oferecemos um estudo empírico com duas bases de dados, uma americana e outra brasileira. Para isolarmos o efeito dos métodos de seleção, nós separamos a fase de seleção de ativos da fase de alocação de ativos. Na fase de alocação de ativos, seguimos Liu (2009) e selecionamos a carteira com a menor volatilidade de Tracking Error. Os métodos de seleção utilizados são o backward stepwise selection, o forward stepwise selection e o lasso. Nossos resultados mostram que, no caso brasileiro, a seleção pelo método lasso é o melhor tracker. No caso americano, o lasso apresenta melhor desempenho ajustado pelo risco, porém isto é devido a maiores retornos, não a menor volatilidade, o que não é desejável no nosso caso. Um destaque deste ensaio é que o melhor tracker para o caso americano é uma carteira que utiliza o algoritmo de iteração backward (métodos simples que recebem pouca atenção na literatura)

Partial order label decomposition approaches for melanoma diagnosis

Melanoma is a type of cancer that develops from the pigment-containing cells known as melanocytes. Usually occurring on the skin, early detection and diagnosis is strongly related to survival rates. Melanoma recognition is a challenging task that nowadays is performed by well trained dermatologists who may produce varying diagnosis due to the task complexity. This motivates the development of automated diagnosis tools, in spite of the inherent difficulties (intra-class variation, visual similarity between melanoma and non-melanoma lesions, among others). In the present work, we propose a system combining image analysis and machine learning to detect melanoma presence and severity. The severity is assessed in terms of melanoma thickness, which is measured by the Breslow index. Previous works mainly focus on the binary problem of detecting the presence of the melanoma. However, the system proposed in this paper goes a step further by also considering the stage of the lesion in the classification task. To do so, we extract 100 features that consider the shape, colour, pigment network and texture of the benign and malignant lesions. The problem is tackled as a five-class classification problem, where the first class represents benign lesions, and the remaining four classes represent the different stages of the melanoma (via the Breslow index). Based on the problem definition, we identify the learning setting as a partial order problem, in which the patterns belonging to the different melanoma stages present an order relationship, but where there is no order arrangement with respect to the benign lesions. Under this assumption about the class topology, we design several proposals to exploit this structure and improve data preprocessing. In this sense, we experimentally demonstrate that those proposals exploiting the partial order assumption achieve better performance than 12 baseline nominal and ordinal classifiers (including a deep learning model) which do not consider this partial order. To deal with class imbalance, we additionally propose specific over-sampling techniques that consider the structure of the problem for the creation of synthetic patterns. The experimental study is carried out with clinician-curated images from the Interactive Atlas of Dermoscopy, which eases reproducibility of experiments. Concerning the results obtained, in spite of having augmented the complexity of the classification problem with more classes, the performance of our proposals in the binary problem is similar to the one reported in the literature

Partial order label decomposition approaches for melanoma diagnosis

Melanoma is a type of cancer that develops from the pigment-containing cells known as melanocytes. Usually occurring on the skin, early detection and diagnosis is strongly related to survival rates. Melanoma recognition is a challenging task that nowadays is performed by well trained dermatologists who may produce varying diagnosis due to the task complexity. This motivates the development of automated diagnosis tools, in spite of the inherent difficulties (intra-class variation, visual similarity between melanoma and non-melanoma lesions, among others). In the present work, we propose a system combining image analysis and machine learning to detect melanoma presence and severity. The severity is assessed in terms of melanoma thickness, which is measured by the Breslow index. Previous works mainly focus on the binary problem of detecting the presence of the melanoma. However, the system proposed in this paper goes a step further by also considering the stage of the lesion in the classification task. To do so, we extract 100 features that consider the shape, colour, pigment network and texture of the benign and malignant lesions. The problem is tackled as a five-class classification problem, where the first class represents benign lesions, and the remaining four classes represent the different stages of the melanoma (via the Breslow index). Based on the problem definition, we identify the learning setting as a partial order problem, in which the patterns belonging to the different melanoma stages present an order relationship, but where there is no order arrangement with respect to the benign lesions. Under this assumption about the class topology, we design several proposals to exploit this structure and improve data preprocessing. In this sense, we experimentally demonstrate that those proposals exploiting the partial order assumption achieve better performance than 12 baseline nominal and ordinal classifiers (including a deep learning model) which do not consider this partial order. To deal with class imbalance, we additionally propose specific over-sampling techniques that consider the structure of the problem for the creation of synthetic patterns. The experimental study is carried out with clinician-curated images from the Interactive Atlas of Dermoscopy, which eases reproducibility of experiments. Concerning the results obtained, in spite of having augmented the complexity of the classification problem with more classes, the performance of our proposals in the binary problem is similar to the one reported in the literature

Block-coordinate descent methods and active-set identification for huge-scale problems

Orientadores: Sandra Augusta Santos, Paulo José da Silva e SilvaTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: Neste trabalho desenvolvemos estratégias de identificação das restrições ativas para o método de descenso coordenado por blocos aplicado a problemas de otimização irrestritos, ou em caixas, cuja função objetivo é a soma de uma função suave e outra convexa. Mostramos que, em certas situações, o método tem a capacidade intrínseca de identificação e também apresentamos um exemplo de função identificadora compatível com a simplicidade computacional exigida pelos problemas de porte enorme. Combinando essas estratégias, desenvolvemos um método de descenso coordenado por blocos, denominado \emph{Active BCDM}, que busca explorar as restrições ativas do problema com restrições de caixa ou, no caso irrestrito, de uma reformulação auxiliar relacionada que possui variáveis não negativas. Analisamos nosso método em duas classes de problemas com muita relevância no contexto de otimização de porte enorme: \textit{LASSO} e regressão logística com regularização $\ell_1$. Preparamos uma ampla discussão de resultados numéricos utilizando problemas reais extraídos da literatura. Isso permite a comparação do \emph{Active BCDM} com vários métodos bem estabelecidos e do estado da arte para estes problemas, tanto no caso sequencial quanto no paralelo. Em ambas implementações, a proposta de identificação apresentou desempenho computacional superior aos métodos com os quais foi comparada. Além disso, resultados de convergência global acompanham os algoritmos propostos, reforçando sua consistência e relevância teóricaAbstract: This work is concerned with the development of strategies to identify active constraints for the block-coordinate descent method applied to unconstrained, or box-constrained, optimization problems whose objective function is the sum of a smooth component and a convex one. We show that, under appropriate assumptions, the method has an intrinsic identification capacity. We also present an example of an identification function compatible with the computational simplicity required to address large-scale problems. Combining these strategies, we have developed a block-coordinate descent method, called Active BCDM, which aims to explore the active constraints in box-constrained problems, or, in the unconstrained case, of a related auxiliary reformulation with non negative variables. We analyze the performance of our method for solving two classes of problems with great relevance in the context of huge-scale optimization: LASSO and $\ell_1$-regularized logistic regression. We have prepared an extensive discussion of numerical results using real problems from the literature. This allows the comparison of Active BCDM with several well-established and state-of-the-art methods for such problems, with sequential and parallel implementations. In both implementations, the identification strategy presented better computational performance among the methods under comparison. In addition, global convergence results have been proved for the proposed algorithms, reinforcing their consistency and theoretical relevanceDoutoradoMatematica AplicadaDoutor em Matemática Aplicada2014/14228-6FAPES

Block-coordinate descent methods and active-set identification for huge-scale problems

Orientadores: Sandra Augusta Santos, Paulo José da Silva e SilvaTese (doutorado) - Universidade Estadual de Campinas, Instituto de Matemática Estatística e Computação CientíficaResumo: Neste trabalho desenvolvemos estratégias de identificação das restrições ativas para o método de descenso coordenado por blocos aplicado a problemas de otimização irrestritos, ou em caixas, cuja função objetivo é a soma de uma função suave e outra convexa. Mostramos que, em certas situações, o método tem a capacidade intrínseca de identificação e também apresentamos um exemplo de função identificadora compatível com a simplicidade computacional exigida pelos problemas de porte enorme. Combinando essas estratégias, desenvolvemos um método de descenso coordenado por blocos, denominado \emph{Active BCDM}, que busca explorar as restrições ativas do problema com restrições de caixa ou, no caso irrestrito, de uma reformulação auxiliar relacionada que possui variáveis não negativas. Analisamos nosso método em duas classes de problemas com muita relevância no contexto de otimização de porte enorme: \textit{LASSO} e regressão logística com regularização $\ell_1$. Preparamos uma ampla discussão de resultados numéricos utilizando problemas reais extraídos da literatura. Isso permite a comparação do \emph{Active BCDM} com vários métodos bem estabelecidos e do estado da arte para estes problemas, tanto no caso sequencial quanto no paralelo. Em ambas implementações, a proposta de identificação apresentou desempenho computacional superior aos métodos com os quais foi comparada. Além disso, resultados de convergência global acompanham os algoritmos propostos, reforçando sua consistência e relevância teóricaAbstract: This work is concerned with the development of strategies to identify active constraints for the block-coordinate descent method applied to unconstrained, or box-constrained, optimization problems whose objective function is the sum of a smooth component and a convex one. We show that, under appropriate assumptions, the method has an intrinsic identification capacity. We also present an example of an identification function compatible with the computational simplicity required to address large-scale problems. Combining these strategies, we have developed a block-coordinate descent method, called Active BCDM, which aims to explore the active constraints in box-constrained problems, or, in the unconstrained case, of a related auxiliary reformulation with non negative variables. We analyze the performance of our method for solving two classes of problems with great relevance in the context of huge-scale optimization: LASSO and $\ell_1$-regularized logistic regression. We have prepared an extensive discussion of numerical results using real problems from the literature. This allows the comparison of Active BCDM with several well-established and state-of-the-art methods for such problems, with sequential and parallel implementations. In both implementations, the identification strategy presented better computational performance among the methods under comparison. In addition, global convergence results have been proved for the proposed algorithms, reinforcing their consistency and theoretical relevanceDoutoradoMatematica AplicadaDoutor em Matemática Aplicada2014/14228-6FAPES