Reinforcement Learning Applied to Trading Systems: A Survey

Financial domain tasks, such as trading in market exchanges, are challenging and have long attracted researchers. The recent achievements and the consequent notoriety of Reinforcement Learning (RL) have also increased its adoption in trading tasks. RL uses a framework with well-established formal concepts, which raises its attractiveness in learning profitable trading strategies. However, RL use without due attention in the financial area can prevent new researchers from following standards or failing to adopt relevant conceptual guidelines. In this work, we embrace the seminal RL technical fundamentals, concepts, and recommendations to perform a unified, theoretically-grounded examination and comparison of previous research that could serve as a structuring guide for the field of study. A selection of twenty-nine articles was reviewed under our classification that considers RL's most common formulations and design patterns from a large volume of available studies. This classification allowed for precise inspection of the most relevant aspects regarding data input, preprocessing, state and action composition, adopted RL techniques, evaluation setups, and overall results. Our analysis approach organized around fundamental RL concepts allowed for a clear identification of current system design best practices, gaps that require further investigation, and promising research opportunities. Finally, this review attempts to promote the development of this field of study by facilitating researchers' commitment to standards adherence and helping them to avoid straying away from the RL constructs' firm ground.Comment: 38 page

Fuzzy Logic and Its Uses in Finance: A Systematic Review Exploring Its Potential to Deal with Banking Crises

The major success of fuzzy logic in the field of remote control opened the door to its application in many other fields, including finance. However, there has not been an updated and comprehensive literature review on the uses of fuzzy logic in the financial field. For that reason, this study attempts to critically examine fuzzy logic as an effective, useful method to be applied to financial research and, particularly, to the management of banking crises. The data sources were Web of Science and Scopus, followed by an assessment of the records according to pre-established criteria and an arrangement of the information in two main axes: financial markets and corporate finance. A major finding of this analysis is that fuzzy logic has not yet been used to address banking crises or as an alternative to ensure the resolvability of banks while minimizing the impact on the real economy. Therefore, we consider this article relevant for supervisory and regulatory bodies, as well as for banks and academic researchers, since it opens the door to several new research axes on banking crisis analyses using artificial intelligence techniques

Effects of diversification among assets in an agent-based market model

We extend to the multi-asset case the framework of a discrete time model of a single asset financial market developed in Ghoulmie et al (2005). In particular, we focus on adaptive agents with threshold behavior allocating their resources among two assets. We explore numerically the effect of this diversification as an additional source of complexity in the financial market and we discuss its destabilizing role. We also point out the relevance of these studies for financial decision making.Comment: 12 pages, 5 figures, accepted for publication in the Proceedings of the Complex Systems II Conference at the Australian National University, 4-7 December 2007, Canberra, ACT Australi

Local flexibility market design for aggregators providing multiple flexibility services at distribution network level

This paper presents a general description of local flexibility markets as a market-based management mechanism for aggregators. The high penetration of distributed energy resources introduces new flexibility services like prosumer or community self-balancing, congestion management and time-of-use optimization. This work is focused on the flexibility framework to enable multiple participants to compete for selling or buying flexibility. In this framework, the aggregator acts as a local market operator and supervises flexibility transactions of the local energy community. Local market participation is voluntary. Potential flexibility stakeholders are the distribution system operator, the balance responsible party and end-users themselves. Flexibility is sold by means of loads, generators, storage units and electric vehicles. Finally, this paper presents needed interactions between all local market stakeholders, the corresponding inputs and outputs of local market operation algorithms from participants and a case study to highlight the application of the local flexibility market in three scenarios. The local market framework could postpone grid upgrades, reduce energy costs and increase distribution grids’ hosting capacity.Postprint (published version

From market games to real-world markets

This paper uses the development of multi-agent market models to present a unified approach to the joint questions of how financial market movements may be simulated, predicted, and hedged against. We examine the effect of different market clearing mechanisms and show that an out-of-equilibrium clearing process leads to dynamics that closely resemble real financial movements. We then show that replacing the `synthetic' price history used by these simulations with data taken from real financial time-series leads to the remarkable result that the agents can collectively learn to identify moments in the market where profit is attainable. We then employ the formalism of Bouchaud and Sornette in conjunction with agent based models to show that in general risk cannot be eliminated from trading with these models. We also show that, in the presence of transaction costs, the risk of option writing is greatly increased. This risk, and the costs, can however be reduced through the use of a delta-hedging strategy with modified, time-dependent volatility structure.Comment: Presented at APFA2 (Liege) July 2000. Proceedings: Eur. Phys. J. B Latex file + 10 .ps figs. [email protected]

APPLICATIONS: Financial risk and financial Risk Management Technology (RMT): Issues and advances

Methods for sound risk management are of increasing interest among Wall Street investment banking and brokerage firms in the aftermath of the October 1987 crash of the stock market. As the knowledge of advanced technology applications in risk management increases, financial firms are finding innovative ways to use them practically, in order to insulate themselves. The recent development in models, the software and hardware, and the market data to track risk are all considered advances in Risk Management Technology (RMT). -. These advances have affected all three stages of risk management: the identification, the measurement, and the formulation of strategies to control financial risk. This article discusses the advances made in five areas of RMT: communication software, object-oriented programming, parallel processing, neural nets and artificial intelligence. Systems based on any of these areas may be used to add value to the business of a firm. A business value linkage analysis shows how the utility of advanced systems can be measured to justify their costs.Information Systems Working Papers Serie