A survey on financial applications of metaheuristics

Modern heuristics or metaheuristics are optimization algorithms that have been increasingly used during the last decades to support complex decision-making in a number of fields, such as logistics and transportation, telecommunication networks, bioinformatics, finance, and the like. The continuous increase in computing power, together with advancements in metaheuristics frameworks and parallelization strategies, are empowering these types of algorithms as one of the best alternatives to solve rich and real-life combinatorial optimization problems that arise in a number of financial and banking activities. This article reviews some of the works related to the use of metaheuristics in solving both classical and emergent problems in the finance arena. A non-exhaustive list of examples includes rich portfolio optimization, index tracking, enhanced indexation, credit risk, stock investments, financial project scheduling, option pricing, feature selection, bankruptcy and financial distress prediction, and credit risk assessment. This article also discusses some open opportunities for researchers in the field, and forecast the evolution of metaheuristics to include real-life uncertainty conditions into the optimization problems being considered.This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (TRA2013-48180-C3-P, TRA2015-71883-REDT), FEDER, and the Universitat Jaume I mobility program (E-2015-36)

Complex stock trading strategy based on particle swarm optimization

Technical Session 1B - Advanced Algorithmic Trading – I: no. 41Trading rules have been utilized in the stock market to make profit for more than a century. However, only using a single trading rule may not be sufficient to predict the stock price trend accurately. Although some complex trading strategies combining various classes of trading rules have been proposed in the literature, they often pick only one rule for each class, which may lose valuable information from other rules in the same class. In this paper, a complex stock trading strategy, namely weight reward strategy (WRS), is proposed. WRS combines the two most popular classes of trading rules-moving average (MA) and trading range break-out (TRB). For both MA and TRB, WRS includes different combinations of the rule parameters to get a universe of 140 component trading rules in all. Each component rule is assigned a start weight and a reward/penalty mechanism based on profit is proposed to update these rules’ weights over time. To determine the best parameter values of WRS, we employ an improved time variant Particle Swarm Optimization (PSO) algorithm with the objective of maximizing the annual net profit generated by WRS. The experiments show that our proposed WRS optimized by PSO outperforms the best moving average and trading range break-out rules.postprin

Learning enhancement of radial basis function network with particle swarm optimization

Back propagation (BP) algorithm is the most common technique in Artificial Neural Network (ANN) learning, and this includes Radial Basis Function Network. However, major disadvantages of BP are its convergence rate is relatively slow and always being trapped at the local minima. To overcome this problem, Particle Swarm Optimization (PSO) has been implemented to enhance ANN learning to increase the performance of network in terms of convergence rate and accuracy. In Back Propagation Radial Basis Function Network (BP-RBFN), there are many elements to be considered. These include the number of input nodes, hidden nodes, output nodes, learning rate, bias, minimum error and activation/transfer functions. These elements will affect the speed of RBF Network learning. In this study, Particle Swarm Optimization (PSO) is incorporated into RBF Network to enhance the learning performance of the network. Two algorithms have been developed on error optimization for Back Propagation of Radial Basis Function Network (BP-RBFN) and Particle Swarm Optimization of Radial Basis Function Network (PSO-RBFN) to seek and generate better network performance. The results show that PSO-RBFN give promising outputs with faster convergence rate and better classifications compared to BP-RBFN

Nature inspired computational intelligence for financial contagion modelling

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Financial contagion refers to a scenario in which small shocks, which initially affect only a few financial institutions or a particular region of the economy, spread to the rest of the financial sector and other countries whose economies were previously healthy. This resembles the “transmission” of a medical disease. Financial contagion happens both at domestic level and international level. At domestic level, usually the failure of a domestic bank or financial intermediary triggers transmission by defaulting on inter-bank liabilities, selling assets in a fire sale, and undermining confidence in similar banks. An example of this phenomenon is the failure of Lehman Brothers and the subsequent turmoil in the US financial markets. International financial contagion happens in both advanced economies and developing economies, and is the transmission of financial crises across financial markets. Within the current globalise financial system, with large volumes of cash flow and cross-regional operations of large banks and hedge funds, financial contagion usually happens simultaneously among both domestic institutions and across countries. There is no conclusive definition of financial contagion, most research papers study contagion by analyzing the change in the variance-covariance matrix during the period of market turmoil. King and Wadhwani (1990) first test the correlations between the US, UK and Japan, during the US stock market crash of 1987. Boyer (1997) finds significant increases in correlation during financial crises, and reinforces a definition of financial contagion as a correlation changing during the crash period. Forbes and Rigobon (2002) give a definition of financial contagion. In their work, the term interdependence is used as the alternative to contagion. They claim that for the period they study, there is no contagion but only interdependence. Interdependence leads to common price movements during periods both of stability and turmoil. In the past two decades, many studies (e.g. Kaminsky et at., 1998; Kaminsky 1999) have developed early warning systems focused on the origins of financial crises rather than on financial contagion. Further authors (e.g. Forbes and Rigobon, 2002; Caporale et al, 2005), on the other hand, have focused on studying contagion or interdependence. In this thesis, an overall mechanism is proposed that simulates characteristics of propagating crisis through contagion. Within that scope, a new co-evolutionary market model is developed, where some of the technical traders change their behaviour during crisis to transform into herd traders making their decisions based on market sentiment rather than underlying strategies or factors. The thesis focuses on the transformation of market interdependence into contagion and on the contagion effects. The author first build a multi-national platform to allow different type of players to trade implementing their own rules and considering information from the domestic and a foreign market. Traders’ strategies and the performance of the simulated domestic market are trained using historical prices on both markets, and optimizing artificial market’s parameters through immune - particle swarm optimization techniques (I-PSO). The author also introduces a mechanism contributing to the transformation of technical into herd traders. A generalized auto-regressive conditional heteroscedasticity - copula (GARCH-copula) is further applied to calculate the tail dependence between the affected market and the origin of the crisis, and that parameter is used in the fitness function for selecting the best solutions within the evolving population of possible model parameters, and therefore in the optimization criteria for contagion simulation. The overall model is also applied in predictive mode, where the author optimize in the pre-crisis period using data from the domestic market and the crisis-origin foreign market, and predict in the crisis period using data from the foreign market and predicting the affected domestic market

From metaheuristics to learnheuristics: Applications to logistics, finance, and computing

Un gran nombre de processos de presa de decisions en sectors estratègics com el transport i la producció representen problemes NP-difícils. Sovint, aquests processos es caracteritzen per alts nivells d'incertesa i dinamisme. Les metaheurístiques són mètodes populars per a resoldre problemes d'optimització difícils en temps de càlcul raonables. No obstant això, sovint assumeixen que els inputs, les funcions objectiu, i les restriccions són deterministes i conegudes. Aquests constitueixen supòsits forts que obliguen a treballar amb problemes simplificats. Com a conseqüència, les solucions poden conduir a resultats pobres. Les simheurístiques integren la simulació a les metaheurístiques per resoldre problemes estocàstics d'una manera natural. Anàlogament, les learnheurístiques combinen l'estadística amb les metaheurístiques per fer front a problemes en entorns dinàmics, en què els inputs poden dependre de l'estructura de la solució. En aquest context, les principals contribucions d'aquesta tesi són: el disseny de les learnheurístiques, una classificació dels treballs que combinen l'estadística / l'aprenentatge automàtic i les metaheurístiques, i diverses aplicacions en transport, producció, finances i computació.Un gran número de procesos de toma de decisiones en sectores estratégicos como el transporte y la producción representan problemas NP-difíciles. Frecuentemente, estos problemas se caracterizan por altos niveles de incertidumbre y dinamismo. Las metaheurísticas son métodos populares para resolver problemas difíciles de optimización de manera rápida. Sin embargo, suelen asumir que los inputs, las funciones objetivo y las restricciones son deterministas y se conocen de antemano. Estas fuertes suposiciones conducen a trabajar con problemas simplificados. Como consecuencia, las soluciones obtenidas pueden tener un pobre rendimiento. Las simheurísticas integran simulación en metaheurísticas para resolver problemas estocásticos de una manera natural. De manera similar, las learnheurísticas combinan aprendizaje estadístico y metaheurísticas para abordar problemas en entornos dinámicos, donde los inputs pueden depender de la estructura de la solución. En este contexto, las principales aportaciones de esta tesis son: el diseño de las learnheurísticas, una clasificación de trabajos que combinan estadística / aprendizaje automático y metaheurísticas, y varias aplicaciones en transporte, producción, finanzas y computación.A large number of decision-making processes in strategic sectors such as transport and production involve NP-hard problems, which are frequently characterized by high levels of uncertainty and dynamism. Metaheuristics have become the predominant method for solving challenging optimization problems in reasonable computing times. However, they frequently assume that inputs, objective functions and constraints are deterministic and known in advance. These strong assumptions lead to work on oversimplified problems, and the solutions may demonstrate poor performance when implemented. Simheuristics, in turn, integrate simulation into metaheuristics as a way to naturally solve stochastic problems, and, in a similar fashion, learnheuristics combine statistical learning and metaheuristics to tackle problems in dynamic environments, where inputs may depend on the structure of the solution. The main contributions of this thesis include (i) a design for learnheuristics; (ii) a classification of works that hybridize statistical and machine learning and metaheuristics; and (iii) several applications for the fields of transport, production, finance and computing

A superior active portfolio optimization model for stock exchange

Due to the vast number of stocks and the multiple appearances of developing investment portfolios, investors in the financial market face multiple investment opportunities. In this regard, the investor task becomes extremely difficult as investors define their preferences for expected return and the amount to which they want to avoid potential investment risks. This research attempts to design active portfolios that outperform the performance of the appropriate market index. To achieve this aim, technical analysis and optimization procedures were used based on a hybrid model. It combines the strong features of the Markowitz model with the General Reduced Gradient (GRG) algorithm to maintain a good compromise between diversification and exploitation. The proposed model is used to construct an active portfolio optimization model for the Iraq Stock Exchange (ISX) for the period from January 2010 to February 2020. This is applied to all 132 companies registered on the exchange. In addition to the market portfolio, two methods, namely, Equal Weight (EW) and Markowitz were used to generate active portfolios to compare the research findings. After a thorough review based on the Sharpe ratio criterion, the suggested model demonstrated its robustness, resulting in maximizing earnings with low risks

Optimization of Aggregators Energy Resources considering Local Markets and Electric Vehicle Penetration

O sector elétrico tem vindo a evoluir ao longo do tempo. Esta situação deve-se ao facto de surgirem novas metodologias para lidarem com a elevada penetração dos recursos energéticos distribuídos (RED), principalmente veículos elétricos (VEs). Neste caso, a gestão dos recursos energéticos tornou-se mais proeminente devido aos avanços tecnológicos que estão a ocorrer, principalmente no contexto das redes inteligentes. Este facto torna-se importante, devido à incerteza decorrente deste tipo de recursos. Para resolver problemas que envolvem variabilidade, os métodos baseados na inteligência computacional estão a se tornar os mais adequados devido à sua fácil implementação e baixo esforço computacional, mais precisamente para o caso tratado na tese, algoritmos de computação evolucionária (CE). Este tipo de algoritmo tenta imitar o comportamento observado na natureza. Ao contrário dos métodos determinísticos, a CEé tolerante à incerteza; ou seja, é adequado para resolver problemas relacionados com os sistemas energéticos. Estes sistemas são geralmente de grandes dimensões, com um número crescente de variáveis e restrições. Aqui a IC permite obter uma solução quase ótima em tempo computacional aceitável com baixos requisitos de memória. O principal objetivo deste trabalho foi propor um modelo para a programação dos recursos energéticos dos recursos dedicados para o contexto intradiário, para a hora seguinte, partindo inicialmente da programação feita para o dia seguinte, ou seja, 24 horas para o dia seguinte. Esta programação é feita por cada agregador (no total cinco) através de meta-heurísticas, com o objetivo de minimizar os custos ou maximizar os lucros. Estes agregadores estão inseridos numa cidade inteligente com uma rede de distribuição de 13 barramentos com elevada penetração de RED, principalmente energia renovável e VEs (2000 VEs são considerados nas simulações). Para modelar a incerteza associada ao RED e aos preços de mercado, vários cenários são gerados através da simulação de Monte Carlo usando as funções de distribuição de probabilidade de erros de previsão, neste caso a função de distribuição normal para o dia seguinte. No que toca à incerteza no modelo para a hora seguinte, múltiplos cenários são gerados a partir do cenário com maior probabilidade do dia seguinte. Neste trabalho, os mercados locais de eletricidade são também utilizados como estratégia para satisfazer a equação do balanço energético onde os agregadores vão para vender o excesso de energia ou comprar para satisfazer o consumo. Múltiplas metaheurísticas de última geração são usadas para fazer este escalonamento, nomeadamente Differential Evolution (DE), Hybrid-Adaptive DE with Decay function (HyDE-DF), DE with Estimation of Distribution Algorithm (DEEDA), Cellular Univariate Marginal Distribution Algorithm with Normal-Cauchy Distribution (CUMDANCauchy++), Hill Climbing to Ring Cellular Encode-Decode UMDA (HC2RCEDUMDA). Os resultados mostram que o modelo proposto é eficaz para os múltiplos agregadores com variações de custo na sua maioria abaixo dos 5% em relação ao dia seguinte, exceto para o agregador e de VEs. É também aplicado um teste Wilcoxon para comparar o desempenho do algoritmo CUMDANCauchy++ com as restantes meta-heurísticas. O CUMDANCauchy++ mostra resultados competitivos tendo melhor performance que todos os algoritmos para todos os agregadores exceto o DEEDA que apresenta resultados semelhantes. Uma estratégia de aversão ao risco é implementada para um agregador no contexto do dia seguinte para se obter uma solução mais segura e robusta. Os resultados mostram um aumento de quase 4% no investimento, mas uma redução de até 14% para o custo dos piores cenários.The electrical sector has been evolving. This situation is because new methodologies emerge to deal with the high penetration of distributed energy resources (DER), mainly electric vehicles (EVs). In this case, energy resource management has become increasingly prominent due to the technological advances that are taking place, mainly in the context of smart grids. This factor becomes essential due to the uncertainty of this type of resource. To solve problems involving variability, methods based on computational intelligence (CI) are becoming the most suitable because of their easy implementation and low computational effort, more precisely for the case treated in this thesis, evolutionary computation (EC) algorithms. This type of algorithm tries to mimic behavior observed in nature. Unlike deterministic methods, the EC is tolerant of uncertainty, and thus it is suitable for solving problems related to energy systems. These systems are usually of high dimensions, with an increased number of variables and restrictions. Here the CI allows obtaining a near-optimal solution in good computational time with low memory requirements. This work's main objective is to propose a model for the energy resource scheduling of the dedicated resources for the intraday context, for the our-ahead, starting initially from the scheduling done for the day ahead, that is, 24 hours for the next day. This scheduling is done by each aggregator (in total five) through metaheuristics to minimize the costs or maximize the profits. These aggregators are inserted in a smart city with a distribution network of 13 buses with a high penetration of DER, mainly renewable energy and EVs (2000 EVs are considered in the simulations). Several scenarios are generated through Monte Carlo Simulation using the forecast errors' probability distribution functions, the normal distribution function for the day-ahead to model the uncertainty associated with DER and market prices. Multiple scenarios are developed through the highest probability scenario from the day-ahead when it comes to intraday uncertainty. In this work, local electricity markets are used as a mechanism to satisfy the energy balance equation where each aggregator can sell the excess of energy or buy more to meet the demand. Several recent and modern metaheuristics are used to solve the proposed problems in the thesis, namely Differential Evolution (DE), Hybrid-Adaptive DE with Decay function (HyDE-DF), DE with Estimation of Distribution Algorithm (DEEDA), Cellular Univariate Marginal Distribution Algorithm with NormalCauchy Distribution (CUMDANCauchy++), Hill Climbing to Ring Cellular Encode-Decode UMDA (HC2RCEDUMDA). Results show that the proposed model is effective for the multiple aggregators. The metaheuristics present satisfactory results and mostly less than 5% variation in costs from the day-ahead except for the EV aggregator. A Wilcoxon test is also applied to compare the performance of the CUMDANCauchy++ algorithm with the remaining metaheuristics. CUMDANCauchy++ shows competitive results beating all algorithms in all aggregators except for DEEDA, which presents similar results. A risk aversion strategy is implemented for an aggregator in the day-ahead context to get a safer and more robust solution. Results show an increase of nearly 4% in day-ahead cost but a reduction of up to 14% of worst scenario cost

The Stock Exchange Prediction using Machine Learning Techniques: A Comprehensive and Systematic Literature Review

This literature review identifies and analyzes research topic trends, types of data sets, learning algorithm, methods improvements, and frameworks used in stock exchange prediction. A total of 81 studies were investigated, which were published regarding stock predictions in the period January 2015 to June 2020 which took into account the inclusion and exclusion criteria. The literature review methodology is carried out in three major phases: review planning, implementation, and report preparation, in nine steps from defining systematic review requirements to presentation of results. Estimation or regression, clustering, association, classification, and preprocessing analysis of data sets are the five main focuses revealed in the main study of stock prediction research. The classification method gets a share of 35.80% from related studies, the estimation method is 56.79%, data analytics is 4.94%, the rest is clustering and association is 1.23%. Furthermore, the use of the technical indicator data set is 74.07%, the rest are combinations of datasets. To develop a stock prediction model 48 different methods have been applied, 9 of the most widely applied methods were identified. The best method in terms of accuracy and also small error rate such as SVM, DNN, CNN, RNN, LSTM, bagging ensembles such as RF, boosting ensembles such as XGBoost, ensemble majority vote and the meta-learner approach is ensemble Stacking. Several techniques are proposed to improve prediction accuracy by combining several methods, using boosting algorithms, adding feature selection and using parameter and hyper-parameter optimization

Evolutionary multi-objective optimization in investment portfolio management

Ph.DDOCTOR OF PHILOSOPH

Machine learning in stock indices trading and pairs trading

This thesis focuses on two fields of machine learning in quantitative trading. The first field uses machine learning to forecast financial time series (Chapters 2 and 3), and then builds a simple trading strategy based on the forecast results. The second (Chapter 4) applies machine learning to optimize decision-making for pairs trading. In Chapter 2, a hybrid Support Vector Machine (SVM) model is proposed and applied to the task of forecasting the daily returns of five popular stock indices in the world, including the S&P500, NKY, CAC, FTSE100 and DAX. The trading application covers the 1997 Asian financial crisis and 2007-2008 global financial crisis. The originality of this work is that the Binary Gravity Search Algorithm (BGSA) is utilized, in order to optimize the parameters and inputs of SVM. The results show that the forecasts made by this model are significantly better than the Random Walk (RW), SVM, best predictors and Buy-and-Hold. The average accuracy of BGSA-SVM for five stock indices is 52.6%-53.1%. The performance of the BGSA-SVM model is not affected by the market crisis, which shows the robustness of this model. In general, this study proves that a profitable trading strategy based on BGSA-SVM prediction can be realized in a real stock market. Chapter 3 focuses on the application of Artificial Neural Networks (ANNs) in forecasting stock indices. It applies the Multi-layer Perceptron (MLP), Convolution Neural Network (CNN) and Long Short-Term Memory (LSTM) neural network to the task of forecasting and trading FTSE100 and INDU indices. The forecasting accuracy and trading performances of MLP, CNN and LSTM are compared under the binary classifications architecture and eight classifications architecture. Then, Chapter 3 combines the forecasts of three ANNs (MLP, CNN and LSTM) by Simple Average, Granger-Ramanathan’s Regression Approach (GRR) and the Least Absolute Shrinkage and Selection Operator (LASSO). Finally, this chapter uses different leverage ratios in trading according to the different daily forecasting probability to improve the trading performance. In Chapter 3, the statistical and trading performances are estimated throughout the period 2000-2018. LSTM slightly outperforms MLP and CNN in terms of average accuracy and average annualized returns. The combination methods do not present improved empirical evidence. Trading using different leverage ratios improves the annualized average return, while the volatility increases. Chapter 4 uses five pairs trading strategies to conduct in-sample training and backtesting on 35 commodities in the major commodity markets from 1980 to 2018. The Distance Method (DIM) and the Co-integration Approach (CA) are used for pairs formation. The Simple Thresholds (ST) strategy, Genetic Algorithm (GA) and Deep Reinforcement Learning (DRL) are employed to determine trading actions. Traditional DIM-ST, CA-ST and CA-DIM-ST are used as benchmark models. The GA is used to optimize the trading thresholds in ST strategy, which is called the CA-GA-ST strategy. Chapter 4 proposes a novel DRL structure for determining trading actions, which replaces the ST decision method. This novel DRL structure is then combined with CA and called the CA-DRL trading strategy. The average annualized returns of the traditional DIM-ST, CA-ST and CA-DIM-ST methods are close to zero. CA-GA-ST uses GA to optimize searches for thresholds. GA selects a smaller range of thresholds, which improves the in-sample performance. However, the average out-of-sample performance only improves slightly, with an average annual return of 1.84% but an increased risk. CA-DRL strategy uses CA to select pairs and then employs DRL to trade the pairs, providing a satisfactory trading performance: the average annualized return reaches 12.49%; the Sharpe Ratio reaches 1.853. Thus, the CA-DRL trading strategy is significantly superior to traditional methods and to CA-GA-ST