Agent-based simulation of electricity markets: a literature review

Liberalisation, climate policy and promotion of renewable energy are challenges to players of the electricity sector in many countries. Policy makers have to consider issues like market power, bounded rationality of players and the appearance of fluctuating energy sources in order to provide adequate legislation. Furthermore the interactions between markets and environmental policy instruments become an issue of increasing importance. A promising approach for the scientific analysis of these developments is the field of agent-based simulation. The goal of this article is to provide an overview of the current work applying this methodology to the analysis of electricity markets. --

Optimal split of orders across liquidity pools: a stochastic algorithm approach

Evolutions of the trading landscape lead to the capability to exchange the same financial instrument on different venues. Because of liquidity issues, the trading firms split large orders across several trading destinations to optimize their execution. To solve this problem we devised two stochastic recursive learning procedures which adjust the proportions of the order to be sent to the different venues, one based on an optimization principle, the other on some reinforcement ideas. Both procedures are investigated from a theoretical point of view: we prove a.s. convergence of the optimization algorithm under some light ergodic (or "averaging") assumption on the input data process. No Markov property is needed. When the inputs are i.i.d. we show that the convergence rate is ruled by a Central Limit Theorem. Finally, the mutual performances of both algorithms are compared on simulated and real data with respect to an "oracle" strategy devised by an "insider" who knows a priori the executed quantities by every venues

Building and investigating generators' bidding strategies in an electricity market

In a deregulated electricity market environment, Generation Companies (GENCOs) compete with each other in the market through spot energy trading, bilateral contracts and other financial instruments. For a GENCO, risk management is among the most important tasks. At the same time, how to maximise its profit in the electricity market is the primary objective of its operations and strategic planning. Therefore, to achieve the best risk-return trade-off, a GENCO needs to determine how to allocate its assets. This problem is also called portfolio optimization. This dissertation presents advanced techniques for generator strategic bidding, portfolio optimization, risk assessment, and a framework for system adequacy optimisation and control in an electricity market environment. Most of the generator bidding related problems can be regarded as complex optimisation problems. In this dissertation, detailed discussions of optimisation methods are given and a number of approaches are proposed based on heuristic global optimisation algorithms for optimisation purposes. The increased level of uncertainty in an electricity market can result in higher risk for market participants, especially GENCOs, and contribute significantly to the drivers for appropriate bidding and risk management tasks for GENCOs in the market. Accordingly, how to build an optimal bidding strategy considering market uncertainty is a fundamental task for GENCOs. A framework of optimal bidding strategy is developed out of this research. To further enhance the effectiveness of the optimal bidding framework; a Support Vector Machine (SVM) based method is developed to handle the incomplete information of other generators in the market, and therefore form a reliable basis for a particular GENCO to build an optimal bidding strategy. A portfolio optimisation model is proposed to maximise the return and minimise the risk of a GENCO by optimally allocating the GENCO's assets among different markets, namely spot market and financial market. A new market pnce forecasting framework is given In this dissertation as an indispensable part of the overall research topic. It further enhances the bidding and portfolio selection methods by providing more reliable market price information and therefore concludes a rather comprehensive package for GENCO risk management in a market environment. A detailed risk assessment method is presented to further the price modelling work and cover the associated risk management practices in an electricity market. In addition to the issues stemmed from the individual GENCO, issues from an electricity market should also be considered in order to draw a whole picture of a GENCO's risk management. In summary, the contributions of this thesis include: 1) a framework of GENCO strategic bidding considering market uncertainty and incomplete information from rivals; 2) a portfolio optimisation model achieving best risk-return trade-off; 3) a FIA based MCP forecasting method; and 4) a risk assessment method and portfolio evaluation framework quantifying market risk exposure; through out the research, real market data and structure from the Australian NEM are used to validate the methods. This research has led to a number of publications in book chapters, journals and refereed conference proceedings

Reinforcement Learning Applied to Trading Systems: A Survey

Financial domain tasks, such as trading in market exchanges, are challenging and have long attracted researchers. The recent achievements and the consequent notoriety of Reinforcement Learning (RL) have also increased its adoption in trading tasks. RL uses a framework with well-established formal concepts, which raises its attractiveness in learning profitable trading strategies. However, RL use without due attention in the financial area can prevent new researchers from following standards or failing to adopt relevant conceptual guidelines. In this work, we embrace the seminal RL technical fundamentals, concepts, and recommendations to perform a unified, theoretically-grounded examination and comparison of previous research that could serve as a structuring guide for the field of study. A selection of twenty-nine articles was reviewed under our classification that considers RL's most common formulations and design patterns from a large volume of available studies. This classification allowed for precise inspection of the most relevant aspects regarding data input, preprocessing, state and action composition, adopted RL techniques, evaluation setups, and overall results. Our analysis approach organized around fundamental RL concepts allowed for a clear identification of current system design best practices, gaps that require further investigation, and promising research opportunities. Finally, this review attempts to promote the development of this field of study by facilitating researchers' commitment to standards adherence and helping them to avoid straying away from the RL constructs' firm ground.Comment: 38 page

Learning To Play The Trading Game

Can we train a stock trading bot that can take decisions in high-entropy envi- ronments like stock markets to generate profits based on some optimal policy? Can we further extend this learning for any general trading problem? Quantitative Al- gorithms are responsible for more than 75% of the stock trading around the world. Creating a stock market prediction model is comparatively easy. But creating a prof- itable prediction model is still considered as a challenging task in the field of machine learning and deep learning due to the unpredictability of the financial markets. Us- ing biologically inspired computing techniques of reinforcement learning (RL) and artificial neural networks(ANN), this project attempts to train an agent who takes decisions based on the optimal decision policies learned. Different existing RL tech- niques and their slightly modified variants will be used to train the agent, and the trained model is then tested against different stock prices and also stock portfolio settings to see if the agent has learned the rules of the game and can it act optimally irrespective of the testing data provided. This work aims to provide general users with simple recommendations about the possible investment decisions of selected stocks in the portfolio. Results of the implemented approaches do seem to work somewhat well on specific periods of stock market time series, but they are observed to be fragile. Selected strategies do not guarantee similar results on all historical time-periods, nor they are guaranteed to provide exceptional performance on unpredictable future stock market time-series data

Application of deep reinforcement learning in stock trading strategies and stock forecasting

The role of the stock market across the overall financial market is indispensable. The way to acquire practical trading signals in the transaction process to maximize the benefits is a problem that has been studied for a long time. This paper put forward a theory of deep reinforcement learning in the stock trading decisions and stock price prediction, the reliability and availability of the model are proved by experimental data, and the model is compared with the traditional model to prove its advantages. From the point of view of stock market forecasting and intelligent decision-making mechanism, this paper proves the feasibility of deep reinforcement learning in financial markets and the credibility and advantages of strategic decision-making