A survey on financial applications of metaheuristics

Modern heuristics or metaheuristics are optimization algorithms that have been increasingly used during the last decades to support complex decision-making in a number of fields, such as logistics and transportation, telecommunication networks, bioinformatics, finance, and the like. The continuous increase in computing power, together with advancements in metaheuristics frameworks and parallelization strategies, are empowering these types of algorithms as one of the best alternatives to solve rich and real-life combinatorial optimization problems that arise in a number of financial and banking activities. This article reviews some of the works related to the use of metaheuristics in solving both classical and emergent problems in the finance arena. A non-exhaustive list of examples includes rich portfolio optimization, index tracking, enhanced indexation, credit risk, stock investments, financial project scheduling, option pricing, feature selection, bankruptcy and financial distress prediction, and credit risk assessment. This article also discusses some open opportunities for researchers in the field, and forecast the evolution of metaheuristics to include real-life uncertainty conditions into the optimization problems being considered.This work has been partially supported by the Spanish Ministry of Economy and Competitiveness (TRA2013-48180-C3-P, TRA2015-71883-REDT), FEDER, and the Universitat Jaume I mobility program (E-2015-36)

Heuristic Strategies in Finance – An Overview

This paper presents a survey on the application of heuristic optimization techniques in the broad field of finance. Heuristic algorithms have been extensively used to tackle complex financial problems, which traditional optimization techniques cannot efficiently solve. Heuristic optimization techniques are suitable for non-linear and non-convex multi-objective optimization problems. Due to their stochastic features and their ability to iteratively update candidate solutions, heuristics can explore the entire search space and reliably approximate the global optimum. This overview reviews the main heuristic strategies and their application to portfolio selection, model estimation, model selection and financial clustering.finance, heuristic optimization techniques, portfolio management, model selection, model estimation, clustering

A Fresh Green Index in the World: Building and optimizing a Vegan and Sustainable Index Fund using a Genetic Algorithm and a Heuristic Local Search

Dissertation presented as the partial requirement for obtaining a Master's degree in Statistics and Information Management, specialization in Risk Analysis and ManagementThe curiosity of investors regarding Environmental, Social and Governance (ESG) factors has seen a growth in the last few years (Alcoforado, 2016), as the world faces some of its biggest problems to date, such as Climate Change and Ecological Collapse. As these issues are not to be taken lightly, individuals have started to act, in the hopes of creating a ‘greener’ world. As individuals hope to align with principles such as Sustainability and Veganism, the proposed project hopes to build a Vegan and Sustainable Index Fund, as “An investment is not an investment if it is destroying our planet.” (Shiva, 2017). The aim of the proposed work is, consequently, to build and optimize an Industry and Geographical diversified Index Fund, using a Genetic Algorithm (GA), demonstrating this through the incorporation of Vegan and Sustainable companies, in addition to the global top-50 ESG ranked firms. Index Funds, which are mutual or Exchange-Traded Funds (ETF), are known to be passively managed portfolios, which have been broadly used in hedge trading (Orito, Inoguchi, & Yamamoto, 2008). This study uses historical data from Vegan, Sustainable and ESG-ranked companies as sample data, replacing traditional optimization methods using a Genetic Algorithm. The GA method was applied to a sample of 61 assets, regarding vegan and sustainable companies, further obtaining a well-diversified and non-centred asset allocation. The obtained results confirm the possible efficiency of genetic algorithms, given their high-speed convergence towards a better solution. A few functions were presented in the algorithm, for example the penalty function method, to perform portfolio optimization which expects to maximize profits and minimize risks. Some flaws have been identified in regard to the method applied

From metaheuristics to learnheuristics: Applications to logistics, finance, and computing

Un gran nombre de processos de presa de decisions en sectors estratègics com el transport i la producció representen problemes NP-difícils. Sovint, aquests processos es caracteritzen per alts nivells d'incertesa i dinamisme. Les metaheurístiques són mètodes populars per a resoldre problemes d'optimització difícils en temps de càlcul raonables. No obstant això, sovint assumeixen que els inputs, les funcions objectiu, i les restriccions són deterministes i conegudes. Aquests constitueixen supòsits forts que obliguen a treballar amb problemes simplificats. Com a conseqüència, les solucions poden conduir a resultats pobres. Les simheurístiques integren la simulació a les metaheurístiques per resoldre problemes estocàstics d'una manera natural. Anàlogament, les learnheurístiques combinen l'estadística amb les metaheurístiques per fer front a problemes en entorns dinàmics, en què els inputs poden dependre de l'estructura de la solució. En aquest context, les principals contribucions d'aquesta tesi són: el disseny de les learnheurístiques, una classificació dels treballs que combinen l'estadística / l'aprenentatge automàtic i les metaheurístiques, i diverses aplicacions en transport, producció, finances i computació.Un gran número de procesos de toma de decisiones en sectores estratégicos como el transporte y la producción representan problemas NP-difíciles. Frecuentemente, estos problemas se caracterizan por altos niveles de incertidumbre y dinamismo. Las metaheurísticas son métodos populares para resolver problemas difíciles de optimización de manera rápida. Sin embargo, suelen asumir que los inputs, las funciones objetivo y las restricciones son deterministas y se conocen de antemano. Estas fuertes suposiciones conducen a trabajar con problemas simplificados. Como consecuencia, las soluciones obtenidas pueden tener un pobre rendimiento. Las simheurísticas integran simulación en metaheurísticas para resolver problemas estocásticos de una manera natural. De manera similar, las learnheurísticas combinan aprendizaje estadístico y metaheurísticas para abordar problemas en entornos dinámicos, donde los inputs pueden depender de la estructura de la solución. En este contexto, las principales aportaciones de esta tesis son: el diseño de las learnheurísticas, una clasificación de trabajos que combinan estadística / aprendizaje automático y metaheurísticas, y varias aplicaciones en transporte, producción, finanzas y computación.A large number of decision-making processes in strategic sectors such as transport and production involve NP-hard problems, which are frequently characterized by high levels of uncertainty and dynamism. Metaheuristics have become the predominant method for solving challenging optimization problems in reasonable computing times. However, they frequently assume that inputs, objective functions and constraints are deterministic and known in advance. These strong assumptions lead to work on oversimplified problems, and the solutions may demonstrate poor performance when implemented. Simheuristics, in turn, integrate simulation into metaheuristics as a way to naturally solve stochastic problems, and, in a similar fashion, learnheuristics combine statistical learning and metaheuristics to tackle problems in dynamic environments, where inputs may depend on the structure of the solution. The main contributions of this thesis include (i) a design for learnheuristics; (ii) a classification of works that hybridize statistical and machine learning and metaheuristics; and (iii) several applications for the fields of transport, production, finance and computing

Partial Index Tracking: A Meta-Learning Approach

Partial index tracking aims to cost effectively replicate the performance of a benchmark index by using a small number of assets. It is usually formulated as a regression problem, but solving it subject to real-world constraints is non-trivial. For example, the common L1 regularised model for sparse regression (i.e., LASSO) is not compatible with those constraints. In this work, we meta-learn a sparse asset selection and weighting strategy that subsequently enables effective partial index tracking by quadratic programming. In particular, we adopt an element-wise L1 norm for sparse regularisation, and meta-learn the weight for each L1 term. Rather than meta-learning a fixed set of hyper-parameters, we meta-learn an inductive predictor for them based on market history, which allows generalisation over time, and even across markets. Experiments are conducted on four indices from different countries, and the empirical results demonstrate the superiority of our method over other baselines. The code is released at https://github.com/qmfin/MetaIndexTracker

Models and Matheuristics for Large-Scale Combinatorial Optimization Problems

Combinatorial optimization deals with efficiently determining an optimal (or at least a good) decision among a finite set of alternatives. In business administration, such combinatorial optimization problems arise in, e.g., portfolio selection, project management, data analysis, and logistics. These optimization problems have in common that the set of alternatives becomes very large as the problem size increases, and therefore an exhaustive search of all alternatives may require a prohibitively long computation time. Moreover, due to their combinatorial nature no closed-form solutions to these problems exist. In practice, a common approach to tackle combinatorial optimization problems is to formulate them as mathematical models and to solve them using a mathematical programming solver (cf., e.g., Bixby et al. 1999, Achterberg et al. 2020). For small-scale problem instances, the mathematical models comprise a manageable number of variables and constraints such that mathematical programming solvers are able to devise optimal solutions within a reasonable computation time. For large-scale problem instances, the number of variables and constraints becomes very large which extends the computation time required to find an optimal solution considerably. Therefore, despite the continuously improving performance of mathematical programming solvers and computing hardware, the availability of mathematical models that are efficient in terms of the number of variables and constraints used is of crucial importance. Another frequently used approach to address combinatorial optimization problems are matheuristics. Matheuristics decompose the considered optimization problem into subproblems, which are then formulated as mathematical models and solved with the help of a mathematical programming solver. Matheuristics are particularly suitable for situations where it is required to find a good, but not necessarily an optimal solution within a short computation time, since the speed of the solution process can be controlled by choosing an appropriate size of the subproblems. This thesis consists of three papers on large-scale combinatorial optimization problems. We consider a portfolio optimization problem in finance, a scheduling problem in project management, and a clustering problem in data analysis. For these problems, we present novel mathematical models that require a relatively small number of variables and constraints, and we develop matheuristics that are based on novel problem-decomposition strategies. In extensive computational experiments, the proposed models and matheuristics performed favorably compared to state-of-the-art models and solution approaches from the literature. In the first paper, we consider the problem of determining a portfolio for an enhanced index-tracking fund. Enhanced index-tracking funds aim to replicate the returns of a particular financial stock-market index as closely as possible while outperforming that index by a small positive excess return. Additionally, we consider various real-life constraints that may be imposed by investors, stock exchanges, or investment guidelines. Since enhanced index-tracking funds are particularly attractive to investors if the index comprises a large number of stocks and thus is well diversified, it is of particular interest to tackle large-scale problem instances. For this problem, we present two matheuristics that consist of a novel construction matheuristic, and two different improvement matheuristics that are based on the concepts of local branching (cf. Fischetti and Lodi 2003) and iterated greedy heuristics (cf., e.g., Ruiz and Stützle 2007). Moreover, both matheuristics are based on a novel mathematical model for which we provide insights that allow to remove numerous redundant variables and constraints. We tested both matheuristics in a computational experiment on problem instances that are based on large stock-market indices with up to 9,427 constituents. It turns out that our matheuristics yield better portfolios than benchmark approaches in terms of out-of-sample risk-return characteristics. In the second paper, we consider the problem of scheduling a set of precedence-related project activities, each of which requiring some time and scarce resources during their execution. For each activity, alternative execution modes are given, which differ in the duration and the resource requirements of the activity. Sought is a start time and an execution mode for each activity, such that all precedence relationships are respected, the required amount of each resource does not exceed its prescribed capacity, and the project makespan is minimized. For this problem, we present two novel mathematical models, in which the number of variables remains constant when the range of the activities' durations and thus also the planning horizon is increased. Moreover, we enhance the performance of the proposed mathematical models by eliminating some symmetric solutions from the search space and by adding some redundant sequencing constraints for activities that cannot be processed in parallel. In a computational experiment based on instances consisting of activities with durations ranging from one up to 260 time units, the proposed models consistently outperformed all reference models from the literature. In the third paper, we consider the problem of grouping similar objects into clusters, where the similarity between a pair of objects is determined by a distance measure based on some features of the objects. In addition, we consider constraints that impose a maximum capacity for the clusters, since the size of the clusters is often restricted in practical clustering applications. Furthermore, practical clustering applications are often characterized by a very large number of objects to be clustered. For this reason, we present a matheuristic based on novel problem-decomposition strategies that are specifically designed for large-scale problem instances. The proposed matheuristic comprises two phases. In the first phase, we decompose the considered problem into a series of generalized assignment problems, and in the second phase, we decompose the problem into subproblems that comprise groups of clusters only. In a computational experiment, we tested the proposed matheuristic on problem instances with up to 498,378 objects. The proposed matheuristic consistently outperformed the state-of-the-art approach on medium- and large-scale instances, while matching the performance for small-scale instances. Although we considered three specific optimization problems in this thesis, the proposed models and matheuristics can be adapted to related optimization problems with only minor modifications. Examples for such related optimization problems are the UCITS-constrained index-tracking problem (cf, e.g., Strub and Trautmann 2019), which consists of determining the portfolio of an investment fund that must comply with regulatory restrictions imposed by the European Union, the multi-site resource-constrained project scheduling problem (cf., e.g., Laurent et al. 2017), which comprises the scheduling of a set of project activities that can be executed at alternative sites, or constrained clustering problems with must-link and cannot-link constraints (cf., e.g., González-Almagro et al. 2020)

Solving cardinality constrained portfolio optimisation problem using genetic algorithms and ant colony optimisation

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonIn this thesis we consider solution approaches for the index tacking problem, in which we aim to reproduces the performance of a market index without purchasing all of the stocks that constitute the index. We solve the problem using three different solution approaches: Mixed Integer Programming (MIP), Genetic Algorithms (GAs), and Ant-colony Optimization (ACO) Algorithm by limiting the number of stocks that can be held. Each index is also assigned with different cardinalities to examine the change to the solution values. All of the solution approaches are tested by considering eight market indices. The smallest data set only consists of 31 stocks whereas the largest data set includes over 2000 stocks. The computational results from the MIP are used as the benchmark to measure the performance of the other solution approaches. The Computational results are presented for different solution approaches and conclusions are given. Finally, we implement post analysis and investigate the best tracking portfolios achieved from the three solution approaches. We summarise the findings of the investigation, and in turn, we further improve some of the algorithms. As the formulations of these problems are mixed-integer linear programs, we use the solver ‘Cplex’ to solve the problems. All of the programming is coded in AMPL

Efficiency-based index tracking optimization model

We used asset efficiency constraints, measured by the Multifractal-Detrended Fluctuation Analysis (MF-DFA), in a classic index tracking model, obtaining a quadratic programming problem (QP). We form portfolios that seek to replicate the S&P500, Nikkei 225 and Ibovespa index, from the US, Japanese and Brazilian markets, respectively, from 2012 to 2021. Our results indicate that efficiency constraints act in a similar way to a cardinality constraint, reducing the number of assets as the constraint becomes more severe, and this movement results in greater tracking error. We evidenced that using efficiency constraint is an adequate way to reduce the number of assets without having to formulate a problem with greater computational complexity, comparing with other methods already used in the literature, especially in more developed markets, where the general levels of efficiency are higher and, therefore, constraints generate a lower cost in terms of tracking error. We also show that efficiency and liquidity constraints could have different effects on tracking portfolios.Utilizamos restrições de eficiência no nível dos ativos, medidas pela Multifractal-Detrended Fluctuation Analysis (MF-DFA), em um modelo clássico de index tracking, obtendo um problema de programação quadrática (PQ). Formamos carteiras que buscam replicar os índices S&P500, Nikkei 225 e Ibovespa, dos mercados norte-americano, japonês e brasileiro, respectivamente, de 2012 a 2021. Nossos resultados indicam que as restrições de eficiência atuam de forma semelhante a uma restrição de cardinalidade, reduzindo o número de ativos à medida que a restrição se torna mais severa, e esse movimento resulta em maior tracking error. Evidenciamos que a utilização de restrição de eficiência é uma forma adequada de reduzir o número de ativos sem a necessidade de formular um problema com maior complexidade computacional, comparando com outros métodos já utilizados na literatura, principalmente em mercados mais desenvolvidos, onde os níveis gerais de eficiência são maiores e, portanto, as restrições geram um custo menor em termos de tracking error. Também, mostramos brevemente que as restrições de eficiência e de liquidez podem ter efeitos diferentes em carteiras de index tracking