Forecasting stylised features of electricity prices in the Australian National Electricity Market

This thesis tests whether forecast accuracy improves when models that explicitly capture the stylised features of the Australian National Electricity Market (NEM) are employed to generate predictions. It is believed that by explicitly modelling these features of electricity wholesale spot prices, the accuracy of the price forecast models can be improved when compared to standard alternative. The stylised features identified in data are mean-reversion, sudden short-lived and consecutive jumps and heavy tails. When employing models to capture the stylised features of electricity prices, the models necessarily become more complex and often contain a greater number of parameters which combine to mimic the characteristics observed in the price series. Throughout this thesis an adherence to the principle of parsimony (Makridakis, et al page 609) will be maintained; that is if two models effectively generate the same forecast performance the simpler model will be preferred whether it contains the stylised features or not. This is also known as Occum’s Razor. This investigation is important in terms of a better understanding of what models are more useful has the potential to lead to more accurate price forecasts which may result in less volatility in market prices leading to more efficient markets. Further, by assessing models that capture various stylised features it may be possible to infer the importance of particular features. Given that wholesale prices are a major determinant of how much end users pay for powering their homes and businesses, it is believed that a better understanding of what forecasting models work (and do not) will allow market participants to develop more successful (business) strategies for adjusting supply to meet demand and to assist with the valuation of financial assets as part of risk management. Additionally, a better understanding of the dynamics of electricity prices and its implications for successful forecasting is important for government policy makers, as Government sets the rules that govern the production and distribution of electricity. It is believed that by explicitly modelling the stylised features of electricity wholesale prices, forecast accuracy can be improved upon baseline models commonly used in quantitative finance. This thesis investigates the forecasting ability of two distinct modelling approaches which by construction capture the stylised characteristics of electricity prices. Namely, these are linear continuous time and non-linear modelling methods. The AR-GARCH model is chosen to be the standard approach in forecasting price series (Engle, 2001) and is taken as the benchmark model in this thesis. More specifically, this thesis aims to answer the following research questions: Does the application of continuous-time models in capturing the stylised features of Australian electricity wholesale spot prices improve forecasting ability upon the traditional AR-GARCH model? Does the application of non-linear forecast models in capturing the stylised features of Australian electricity wholesale spot prices improve forecast ability upon traditional AR-GARCH model? The continuous-time models examined in this thesis are; Geometric Brownian Motion (GBM), Mean-Reverting, and Mean-Reverting Jump-Diffusion processes. The inclusion of GBM in this thesis is due to it being the foundation for the Mean-Reverting and Jump-Diffusion models, which are considered in this thesis. Continuous-time models capture some of the main stylised features of electricity prices; Mean-Reverting process captures the mean-reversion (tendency of electricity prices to revert back to its long-term average over time) characteristics of electricity prices whilst Mean-Reverting and Jump-Diffusion process models the sudden jumps prevalent in Australian electricity prices. The models are in order such that each successive model extends the one preceding it. Note that each extension addresses a stylised feature of the data therefore the a priori expectation is that the forecasting performance will improve. The inclusion of the non-linear approach to forecasting Australian electricity prices is performed with the application of a Markov Regime-Switching model and the application of Extreme Value Theory (EVT) into electricity price modelling. The Markov Regime-Switching model is a non-linear modelling tool that is able to capture consecutive spikes prevalent in electricity prices that Mean-Reverting and Jump-Diffusion processes fail to capture. The application of EVT is included in this thesis so that heavy tails present in electricity prices can be adequately captured. Copulas are considered as a unique method that models the dependence structure of data. The forecasts based on the EVT model is built upon the application of Copula functions as these functions model the interdependence of prices within the separate regions of the Australian electricity markets. The models examined in this thesis are: 1. AR(1)-GARCH(1) 2. Geometric Brownian Motion 3. Mean-Reverting Model 4. Mean-Reverting and Jump-Diffusion Model 5. Markov Regime-Switching Model with spike distributions modelled with 6. -Gaussian distribution 7. -Log-Gaussian distribution and, 8. Extreme value Theory and Copula functions Each model under investigation mimics a known characteristic of electricity prices. Comparative performance evaluations of each model investigated in this thesis showed that the benchmark model is providing superior short-term forecasting ability

Ensemble prediction model with expert selection for electricity price forecasting

Forecasting of electricity prices is important in deregulated electricity markets for all of the stakeholders: energy wholesalers, traders, retailers and consumers. Electricity price forecasting is an inherently difficult problem due to its special characteristic of dynamicity and non-stationarity. In this paper, we present a robust price forecasting mechanism that shows resilience towards the aggregate demand response effect and provides highly accurate forecasted electricity prices to the stakeholders in a dynamic environment. We employ an ensemble prediction model in which a group of different algorithms participates in forecasting 1-h ahead the price for each hour of a day. We propose two different strategies, namely, the Fixed Weight Method (FWM) and the Varying Weight Method (VWM), for selecting each hour’s expert algorithm from the set of participating algorithms. In addition, we utilize a carefully engineered set of features selected from a pool of features extracted from the past electricity price data, weather data and calendar data. The proposed ensemble model offers better results than the Autoregressive Integrated Moving Average (ARIMA) method, the Pattern Sequence-based Forecasting (PSF) method and our previous work using Artificial Neural Networks (ANN) alone on the datasets for New York, Australian and Spanish electricity markets

Iberian Energy Market: Spot Price Forecast by Modelling Market Offers

Electricity is a very special commodity since it is economically non-storable, and thus requiring a constant balance between production and consumption. At the corporate level, electricity price forecasts have become a fundamental input to energy companies’ decision making mechanisms [22, 45]. Electric utilities are higly vulnerable to economical crisis, since they generally cannot pass their excess costs on the wholesale market to the retail consumers [77] and, since the price depends on variables like weather (temperature, wind speed, precipitation, etc.) and the intensity of business and everyday activities (on-peak vs. off-peak hours, weekdays vs. weekends, holidays and near-holidays, etc.) it shows specific dynamics not observed in any other market, exhibiting seasonality at the daily, weekly and annual levels, and abrupt, short-lived and generally unanticipated price spikes. These extreme price volatility make price forecasts from a few hours to a few months ahead to become of particular interest to power portfolio managers. An utility company or large industrial consumer who is able to accurately forecast the wholesale prices and it’s volatility, can adjust its bidding strategy and its own production/consumption schedule in order to reduce the risk or maximize the profits in day-ahead trading. In this work I discuss the dynamics of the Iberian electricity day-ahead market (OMIE), review the state-of-the-art forecasting techniques and introduce a new approach to Electricity Price Forecasting, by forecasting the underlying dynamics, the market demand/supply curves. With this method it is possible to predict not only the electricity prices for the next hours, but also the market curves, which can then be used for risk management and a more accurate schedule of generation units. I analyze the model results and benchmark them against other models in the industry.A eletricidade é uma commodity muito especial, uma vez que não é possível armazená-la, e por isso, requer um constante equilíbrio entre a produção e consumo. ao nível empresarial, a previsão de preços de eletricidade tornou-se um input fundamental para os mecanismos de tomada de decisão das companhias [22, 45]. As empresas de eletricidade são altamente vulneráveis a crises económicas, uma vez que, em geral, não conseguem passar os seus custos excessivos para o mercado retalhista [77] e, uma vez que o preço depende de variáveis como meteorologia (temperatura, velocidade do vento, precipitação, etc.) e da intensidade de negócio e das atividades do dia-a-dia (pico vs vazio, dias da semana vs fim-de-semana, feriados e pontes, etc.) apresenta uma dinâmica que não é observada em mais nenhum mercado, com sazonalidade diária, semanal e anual, e com picos de preço abruptos de pouca duração e, em termos gerais, impossíveis de antecipar. Esta volatilidade de preços torna a previsão de preços particularmente interessante para gestores de portfólio, seja a curto ou a longo prazo. Uma companhia de eletricidade ou grande consumidor industrial que seja capaz de prever corretamente os preços do mercado grossista e a sua volatilidade, pode ajustar a estratégia de oferta da sua produção/seu consumo de maneira a reduzir o risco ou maximizar os ganhos no mercado à vista. Neste trabalho abordo a dinâmica do mercado de eletricidade ibérico (Operador de Mercado Iberico - Polo Español (OMIE)), revendo o estado da arte dos métodos de previsão de preços de eletricidade, e introduzo uma nova técnica de previsão de preços de eletricidade, através da previsão da sua dinâmica subjacente, as curvas de mercado da procura e oferta. Com este método é possível prever, não só o preço de eletricidade para as próximas horas, mas também as próprias curvas de oferta, o que pode ser utilizado na gestão de risco ao melhor a capacidade de programar as suas unidades de geração.Os resultados do modelo são analisados e comparados com outros modelos já utilizados na industria

Some Essays on models in the Bond and Energy Markets

The term structure of interest rates plays a fundamental role as an indicator of economy and market trends, as well as a supporting tool for macroeconomic strategies, investment choices or hedging practices. Therefore, the availability of proper techniques to model and predict its dynamics is of crucial importance for players in the financial markets. Along this path, the dissertation initially examined the reliability of parametric and neural network models to fit and predict the term structure of interest rates in emerging markets, focusing on the Brazilian, Russian, Indian, Chines and South African (BRICS) bond markets. The focus on the BRICS is straightforward: the dynamics of their term structures make tricky the application of consolidated yield curve models. In this respect, BRICS yield curve act as stress testers. The study then examined how to apply the above cited models to energy derivatives, focusing the attention on the Natural Gas and Electricity futures, motivated by the existence of similarity. The research was carried out using ad hoc routines, such as the R package "DeRezende.Ferreira", developed by the candidate and now freely downloadable at the Comprehensive R Archive Network (CRAN) repository*, as well as by means of code written in MatLab 2021a - 2022a and Python (3.10.10) using the open-source Keras (2.4.3) library with TensorFlow (2.4.0) as backend. The dissertation consists of four chapters based on published and/or under submission materials. Chapter 1 is an excerpt of the paper • Castello, O.; Resta, M. Modeling the Yield Curve of BRICS Countries: Parametric vs. Machine Learning Techniques. Risks 2022 The work firstly offers a comprehensive analysis of the BRICS bond market and then investigates and compares the abilities of the parametric Five–Factor De Rezende–Ferreira model and Feed–Forward Neural Networks to fit the yield curves. Chapter 2 is again focused on the BRICS market but investigates a methodology to identify optimal time–varying parameters for parametric yield curve models. The work then investigates the ability of this method both for in–sample fitting and out–of–sample prediction. Various forecasting methods are examined: the Univariate Autoregressive process AR(1), the TBATS and the Autoregressive Integrated Moving Average (ARIMA) combined to Nonlinear Autoregressive Neural Networks (NAR–NN). Chapter 3 studies the term structure dynamics in the Natural Gas futures market. This chapter represents an extension of the paper • Castello, O., Resta, M. (2022). Modeling and Forecasting Natural Gas Futures Prices Dynamics: An Integrated Approach. In: Corazza, M., Perna, C., Pizzi, C., Sibillo, M. (eds) Mathematical and Statistical Methods for Actuarial Sciences and Finance. MAF 2022. After showing that the natural gas and bond markets share similar stylized facts, we exploit these findings to examine whether techniques conventionally employed on the bonds market can be effectively used also for accurate in–sample fitting and out–of–sample forecast. We worked at first in–sample and we compared the performance of three models: the Four–Factor Dynamic Nelson–Siegel–Svensson (4F-DNSS), the Five–Factor Dynamic De Rezende–Ferreira (5F–DRF) and the B–Spline. Then, we turned the attention on forecasting, and explored the effectiveness of a hybrid methodology relying on the joint use of 4F–DNSS, 5F–DRF and B–Splines with Nonlinear Autoregressive Neural Networks (NAR–NNs). Empirical study was carried on using the Dutch Title Transfer Facility (TTF) daily futures prices in the period from January 2011 to June 2022 which included also recent market turmoil to validate the overall effectiveness of the framework. Chapter 4 analyzes the predictability of the electricity futures prices term structure with Artificial Neural Networks. Prices time series and futures curves are characterized by high volatility which is a direct consequence of an inelastic demand and of the non–storable nature of the underlying commodity. We analyzed the forecasting power of several neural network models, including Nonlinear Autoregressive (NAR–NNs), NAR with Exogenous Inputs (NARX–NNs), Long Short–Term Memory (LSTM–NNs) and Encoder–Decoder Long Short–Term Memory Neural Networks (ED–LSTM–NNs). We carried out an extensive study of the models predictive capabilities using both the univariate and multivariate setting. Additionally, we explored whether incorporating various exogenous components such as Carbon Emission Certificates (CO2) spot prices, as well as Natural Gas and Coal futures prices can lead to improvements of the models performances. The data of the European Energy Exchange (EEX) power market were adopted to test the models. Chapter 4 concludes. ____________________________ * https://cran.r-project.org/web/packages/DeRezende.Ferreira/index.htm

Building and investigating generators' bidding strategies in an electricity market

In a deregulated electricity market environment, Generation Companies (GENCOs) compete with each other in the market through spot energy trading, bilateral contracts and other financial instruments. For a GENCO, risk management is among the most important tasks. At the same time, how to maximise its profit in the electricity market is the primary objective of its operations and strategic planning. Therefore, to achieve the best risk-return trade-off, a GENCO needs to determine how to allocate its assets. This problem is also called portfolio optimization. This dissertation presents advanced techniques for generator strategic bidding, portfolio optimization, risk assessment, and a framework for system adequacy optimisation and control in an electricity market environment. Most of the generator bidding related problems can be regarded as complex optimisation problems. In this dissertation, detailed discussions of optimisation methods are given and a number of approaches are proposed based on heuristic global optimisation algorithms for optimisation purposes. The increased level of uncertainty in an electricity market can result in higher risk for market participants, especially GENCOs, and contribute significantly to the drivers for appropriate bidding and risk management tasks for GENCOs in the market. Accordingly, how to build an optimal bidding strategy considering market uncertainty is a fundamental task for GENCOs. A framework of optimal bidding strategy is developed out of this research. To further enhance the effectiveness of the optimal bidding framework; a Support Vector Machine (SVM) based method is developed to handle the incomplete information of other generators in the market, and therefore form a reliable basis for a particular GENCO to build an optimal bidding strategy. A portfolio optimisation model is proposed to maximise the return and minimise the risk of a GENCO by optimally allocating the GENCO's assets among different markets, namely spot market and financial market. A new market pnce forecasting framework is given In this dissertation as an indispensable part of the overall research topic. It further enhances the bidding and portfolio selection methods by providing more reliable market price information and therefore concludes a rather comprehensive package for GENCO risk management in a market environment. A detailed risk assessment method is presented to further the price modelling work and cover the associated risk management practices in an electricity market. In addition to the issues stemmed from the individual GENCO, issues from an electricity market should also be considered in order to draw a whole picture of a GENCO's risk management. In summary, the contributions of this thesis include: 1) a framework of GENCO strategic bidding considering market uncertainty and incomplete information from rivals; 2) a portfolio optimisation model achieving best risk-return trade-off; 3) a FIA based MCP forecasting method; and 4) a risk assessment method and portfolio evaluation framework quantifying market risk exposure; through out the research, real market data and structure from the Australian NEM are used to validate the methods. This research has led to a number of publications in book chapters, journals and refereed conference proceedings

A hybrid model for day-ahead electricity price forecasting: Combining fundamental and stochastic modelling

The accurate prediction of short-term electricity prices is vital for effective trading strategies, power plant scheduling, profit maximisation and efficient system operation. However, uncertainties in supply and demand make such predictions challenging. We propose a hybrid model that combines a techno-economic energy system model with stochastic models to address this challenge. The techno-economic model in our hybrid approach provides a deep understanding of the market. It captures the underlying factors and their impacts on electricity prices, which is impossible with statistical models alone. The statistical models incorporate non-techno-economic aspects, such as the expectations and speculative behaviour of market participants, through the interpretation of prices. The hybrid model generates both conventional point predictions and probabilistic forecasts, providing a comprehensive understanding of the market landscape. Probabilistic forecasts are particularly valuable because they account for market uncertainty, facilitating informed decision-making and risk management. Our model delivers state-of-the-art results, helping market participants to make informed decisions and operate their systems more efficiently

Forecasting: theory and practice

Forecasting has always been in the forefront of decision making and planning. The uncertainty that surrounds the future is both exciting and challenging, with individuals and organisations seeking to minimise risks and maximise utilities. The lack of a free-lunch theorem implies the need for a diverse set of forecasting methods to tackle an array of applications. This unique article provides a non-systematic review of the theory and the practice of forecasting. We offer a wide range of theoretical, state-of-the-art models, methods, principles, and approaches to prepare, produce, organise, and evaluate forecasts. We then demonstrate how such theoretical concepts are applied in a variety of real-life contexts, including operations, economics, finance, energy, environment, and social good. We do not claim that this review is an exhaustive list of methods and applications. The list was compiled based on the expertise and interests of the authors. However, we wish that our encyclopedic presentation will offer a point of reference for the rich work that has been undertaken over the last decades, with some key insights for the future of the forecasting theory and practice

A probabilistic forecast methodology for volatile electricity prices in the Australian National Electricity Market

The South Australia region of the Australian National Electricity Market (NEM) displays some of the highest levels of price volatility observed in modern electricity markets. This paper outlines an approach to probabilistic forecasting under these extreme conditions, including spike filtration and several post-processing steps. We propose using quantile regression as an ensemble tool for probabilistic forecasting, with our combined forecasts achieving superior results compared to all constituent models. Within our ensemble framework, we demonstrate that averaging models with varying training length periods leads to a more adaptive model and increased prediction accuracy. The applicability of the final model is evaluated by comparing our median forecasts with the point forecasts available from the Australian NEM operator, with our model outperforming these NEM forecasts by a significant margin.Comment: This manuscript has been accepted for publication in International Journal of Forecastin

Forecasting and Risk Management Techniques for Electricity Markets

This book focuses on the recent development of forecasting and risk management techniques for electricity markets. In addition, we discuss research on new trading platforms and environments using blockchain-based peer-to-peer (P2P) markets and computer agents. The book consists of two parts. The first part is entitled “Forecasting and Risk Management Techniques” and contains five chapters related to weather and electricity derivatives, and load and price forecasting for supporting electricity trading. The second part is entitled “Peer-to-Peer (P2P) Electricity Trading System and Strategy” and contains the following five chapters related to the feasibility and enhancement of P2P energy trading from various aspects