Deep Learning can Replicate Adaptive Traders in a Limit-Order-Book Financial Market

We report successful results from using deep learning neural networks (DLNNs) to learn, purely by observation, the behavior of profitable traders in an electronic market closely modelled on the limit-order-book (LOB) market mechanisms that are commonly found in the real-world global financial markets for equities (stocks & shares), currencies, bonds, commodities, and derivatives. Successful real human traders, and advanced automated algorithmic trading systems, learn from experience and adapt over time as market conditions change; our DLNN learns to copy this adaptive trading behavior. A novel aspect of our work is that we do not involve the conventional approach of attempting to predict time-series of prices of tradeable securities. Instead, we collect large volumes of training data by observing only the quotes issued by a successful sales-trader in the market, details of the orders that trader is executing, and the data available on the LOB (as would usually be provided by a centralized exchange) over the period that the trader is active. In this paper we demonstrate that suitably configured DLNNs can learn to replicate the trading behavior of a successful adaptive automated trader, an algorithmic system previously demonstrated to outperform human traders. We also demonstrate that DLNNs can learn to perform better (i.e., more profitably) than the trader that provided the training data. We believe that this is the first ever demonstration that DLNNs can successfully replicate a human-like, or super-human, adaptive trader operating in a realistic emulation of a real-world financial market. Our results can be considered as proof-of-concept that a DLNN could, in principle, observe the actions of a human trader in a real financial market and over time learn to trade equally as well as that human trader, and possibly better.Comment: 8 pages, 4 figures. To be presented at IEEE Symposium on Computational Intelligence in Financial Engineering (CIFEr), Bengaluru; Nov 18-21, 201

Adaptive Agents and Data Quality in Agent-Based Financial Markets

We present our Agent-Based Market Microstructure Simulation (ABMMS), an Agent-Based Financial Market (ABFM) that captures much of the complexity present in the US National Market System for equities (NMS). Agent-Based models are a natural choice for understanding financial markets. Financial markets feature a constrained action space that should simplify model creation, produce a wealth of data that should aid model validation, and a successful ABFM could strongly impact system design and policy development processes. Despite these advantages, ABFMs have largely remained an academic novelty. We hypothesize that two factors limit the usefulness of ABFMs. First, many ABFMs fail to capture relevant microstructure mechanisms, leading to differences in the mechanics of trading. Second, the simple agents that commonly populate ABFMs do not display the breadth of behaviors observed in human traders or the trading systems that they create. We investigate these issues through the development of ABMMS, which features a fragmented market structure, communication infrastructure with propagation delays, realistic auction mechanisms, and more. As a baseline, we populate ABMMS with simple trading agents and investigate properties of the generated data. We then compare the baseline with experimental conditions that explore the impacts of market topology or meta-reinforcement learning agents. The combination of detailed market mechanisms and adaptive agents leads to models whose generated data more accurately reproduce stylized facts observed in actual markets. These improvements increase the utility of ABFMs as tools to inform design and policy decisions.Comment: 11 pages, 6 figures, and 1 table. Contains 12 pages of supplemental information with 1 figure and 22 table

Market Impact in Trader-Agents:Adding Multi-Level Order-Flow Imbalance-Sensitivity to Automated Trading Systems

Financial markets populated by human traders often exhibit "market impact", where the traders' quote-prices move in the direction of anticipated change, before any transaction has taken place, as an immediate reaction to the arrival of a large (i.e., "block") buy or sell order in the market: e.g., traders in the market know that a block buy order will push the price up, and so they immediately adjust their quote-prices upwards. Most major financial markets now involve many "robot traders", autonomous adaptive software agents, rather than humans. This paper explores how to give such trader-agents a reliable anticipatory sensitivity to block orders, such that markets populated entirely by robot traders also show market-impact effects. In a 2019 publication Church & Cliff presented initial results from a simple deterministic robot trader, ISHV, which exhibits this market impact effect via monitoring a metric of imbalance between supply and demand in the market. The novel contributions of our paper are: (a) we critique the methods used by Church & Cliff, revealing them to be weak, and argue that a more robust measure of imbalance is required; (b) we argue for the use of multi-level order-flow imbalance (MLOFI: Xu et al., 2019) as a better basis for imbalance-sensitive robot trader-agents; and (c) we demonstrate the use of the more robust MLOFI measure in extending ISHV, and also the well-known AA and ZIP trading-agent algorithms (which have both been previously shown to consistently outperform human traders). We demonstrate that the new imbalance-sensitive trader-agents introduced here do exhibit market impact effects, and hence are better-suited to operating in markets where impact is a factor of concern or interest, but do not suffer the weaknesses of the methods used by Church & Cliff. The source-code for our work reported here is freely available on GitHub.Comment: To be presented at the 13th International Conference on Agents and Artificial Intelligence (ICAART2021), Vienna, 4th--6th February 2021. 15 pages; 9 figure

Adaptive-Aggressive Traders Don't Dominate

For more than a decade Vytelingum's Adaptive-Aggressive (AA) algorithm has been recognized as the best-performing automated auction-market trading-agent strategy currently known in the AI/Agents literature; in this paper, we demonstrate that it is in fact routinely outperformed by another algorithm when exhaustively tested across a sufficiently wide range of market scenarios. The novel step taken here is to use large-scale compute facilities to brute-force exhaustively evaluate AA in a variety of market environments based on those used for testing it in the original publications. Our results show that even in these simple environments AA is consistently out-performed by IBM's GDX algorithm, first published in 2002. We summarize here results from more than one million market simulation experiments, orders of magnitude more testing than was reported in the original publications that first introduced AA. A 2019 ICAART paper by Cliff claimed that AA's failings were revealed by testing it in more realistic experiments, with conditions closer to those found in real financial markets, but here we demonstrate that even in the simple experiment conditions that were used in the original AA papers, exhaustive testing shows AA to be outperformed by GDX. We close this paper with a discussion of the methodological implications of our work: any results from previous papers where any one trading algorithm is claimed to be superior to others on the basis of only a few thousand trials are probably best treated with some suspicion now. The rise of cloud computing means that the compute-power necessary to subject trading algorithms to millions of trials over a wide range of conditions is readily available at reasonable cost: we should make use of this; exhaustive testing such as is shown here should be the norm in future evaluations and comparisons of new trading algorithms.Comment: To be published as a chapter in "Agents and Artificial Intelligence" edited by Jaap van den Herik, Ana Paula Rocha, and Luc Steels; forthcoming 2019/2020. 24 Pages, 1 Figure, 7 Table

An Adaptive Dual-level Reinforcement Learning Approach for Optimal Trade Execution

The purpose of this research is to devise a tactic that can closely track the daily cumulative volume-weighted average price (VWAP) using reinforcement learning. Previous studies often choose a relatively short trading horizon to implement their models, making it difficult to accurately track the daily cumulative VWAP since the variations of financial data are often insignificant within the short trading horizon. In this paper, we aim to develop a strategy that can accurately track the daily cumulative VWAP while minimizing the deviation from the VWAP. We propose a method that leverages the U-shaped pattern of intraday stock trade volumes and use Proximal Policy Optimization (PPO) as the learning algorithm. Our method follows a dual-level approach: a Transformer model that captures the overall(global) distribution of daily volumes in a U-shape, and a LSTM model that handles the distribution of orders within smaller(local) time intervals. The results from our experiments suggest that this dual-level architecture improves the accuracy of approximating the cumulative VWAP, when compared to previous reinforcement learning-based models.Comment: Submitted to Expert Systems with Applications (Under 2nd review

BSE:A Minimal Simulation of a Limit-Order-Book Stock Exchange

This paper describes the design, implementation, and successful use of the Bristol Stock Exchange (BSE), a novel minimal simulation of a centralised financial market, based on a Limit Order Book (LOB) such as is common in major stock exchanges. Construction of BSE was motivated by the fact that most of the world's major financial markets have automated, with trading activity that previously was the responsibility of human traders now being performed by high-speed autonomous automated trading systems. Research aimed at understanding the dynamics of this new style of financial market is hampered by the fact that no operational real-world exchange is ever likely to allow experimental probing of that market while it is open and running live, forcing researchers to work primarily from time-series of past trading data. Similarly, university-level education of the engineers who can create next-generation automated trading systems requires that they have hands-on learning experience in a sufficiently realistic teaching environment. BSE as described here addresses both those needs: it has been successfully used for teaching and research in a leading UK university since 2012, and the BSE program code is freely available as open-source on GitHuB.Comment: 10 pages, 6 figures. To appear in Proceedings of 30th European Modelling and Simulation Symposium (EMSS-2018), Budapest, Hungary, September 17-19, 201