Explainable Artificial Intelligence and Deep Learning for Analysis and Forecasting of Complex Time Series: Applications to Electricity Prices

A rapid energy transition from fossil fuel based generation to renewable energy sources is vital for the mitigation of climate change but requires complex market structures to manage the coordination of generation and demand. In particular, the German day-ahead market reacts to short-term forecasts one day prior to delivery and is driven by various external drivers. Its understanding and forecasting are essential for the energy transition as it allows renewable energy operators to make profits and promotes key technologies for a stable grid operation, such as battery storage. In this work, we analyze the German day-ahead electricity market using eXplainable Artificial Intelligence (XAI) and forecast electricity prices using deep neural networks. We investigate the application of SHapley Additive exPlanations (SHAP) to study the driving factors of electricity prices. The dataset includes several power system features such as load or renewable forecasts but also fuel prices. Our analysis suggests that load, wind and solar generation are the central external features driving prices, as expected, wherein wind generation affects prices more than solar generation. Simi- larly, fuel prices also highly affect prices in a nontrivial manner. Moreover, large generation ramps are correlated with high prices due to the limited flexibility of nuclear and lignite plants. Based on the results from the XAI method, we establish Long Short-Term Memory (LSTM) networks to forecast electricity prices. We introduce a probabilistic forecast as output, increas- ing the applicability of the model. The LSTM model is able to outperform models from related works and enables additional applications using the predicted standard deviation

Patterns and correlations in European electricity prices

Electricity markets are central to the coordination of power generation and demand. The European power system is divided into several bidding zones, each having an individual electricity market price. While individual price time series have been intensively studied in recent years, spatiotemporal aspects have received little attention. This article provides a comprehensive data-centric analysis of the patterns and correlations of the European day-ahead electricity prices between 2019 and 2023, characteristically abnormal due to the energy crisis in Europe. We identify the dominant communities of bidding zones and show that spatial differences can be described with very few principal components. Most bidding zones in Continental Europe were brought together during the energy crisis: Correlations increased, and the number of relevant principal components decreased. Opposite effects occur in the Nordic countries and the Iberian Peninsula where correlations decrease and communities fragment.Patterns and correlations in European electricity pricespublishedVersio

Influenza Vaccine Type-Dependent Antibody Response in Patients with Autoimmune Inflammatory Rheumatic Diseases

BACKGROUND: The study aimed to explore influenza antibody response in patients with autoimmune inflammatory rheumatoid diseases (AIIRDs) stratified by the different vaccine types applied in Denmark during the 2018-2019 influenza season.METHODS: Included patients were diagnosed with rheumatoid arthritis, psoriatic arthritis, or spondyloarthritis receiving biological disease-modifying antirheumatic drugs (bDMARDs) with or without conventional synthetic disease-modifying antirheumatic drugs. Influenza vaccination status in the 2018-2019 season and vaccine type received were reviewed in the Denmark. Blood samples were drawn ≥ 14 days post vaccination, and antibody titers were determined by the hemagglutinin inhibition (HAI) assay for the serotypes A/Michigan/H1N1, A/Singapore/H3N2, and B/Colorado included in the influenza vaccines in the 2018-2019 season. An overall serotype HAI geometric mean titer (GMT) was calculated from the 3 serotype-specific HAI titers. An overall serotype HAI GMT ≥ 40 was considered protective.RESULTS: Of the 205 included patients, 105 (51%) had received influenza vaccination. One-quarter of vaccinated patients achieved post-vaccination overall serotype HAI GMT ≥40. For patients vaccinated with Influvac, a significantly higher proportion had HAI titers ≥ 40 for 2 serotypes, namely, A/Michigan/H1N1 and A/Singapore/H3N2, than patients vaccinated with Vaxigrip or VaxigripTetra. The same applied to all serotypes HAI GMT, where significantly more patients who received Influvac achieved postvaccination HAI GMT≥40 versus patients who received Vaxigrip (p=0.02) or VaxigripTetra (p=0.002). The latter outcome was explored in a multivariable logistic regression analysis and remained significant when including the following variables: age, sex, treatment with methotrexate and/or prednisolone, type of influenza vaccine, time interval from vaccination to antibody measurement, and previous vaccination status.CONCLUSION: Influenza antibody levels following vaccination with Influvac in bDMARD-treated patients with AIIRDs were superior to Vaxigrip and VaxigripTetra. Treatment with methotrexate (MTX) did not reduce the antibody response.</p

Understanding electricity prices beyond the merit order principle using explainable AI

Electricity prices in liberalized markets are determined by the supply and demand for electric power, which are in turn driven by various external influences that vary strongly in time. In perfect competition, the merit order principle describes that dispatchable power plants enter the market in the order of their marginal costs to meet the residual load, i.e. the difference of load and renewable generation. Many market models implement this principle to predict electricity prices but typically require certain assumptions and simplifications. In this article, we present an explainable machine learning model for the prices on the German day-ahead market, which substantially outperforms a benchmark model based on the merit order principle. Our model is designed for the ex-post analysis of prices and thus builds on various external features. Using Shapley Additive exPlanation (SHAP) values, we can disentangle the role of the different features and quantify their importance from empiric data. Load, wind and solar generation are most important, as expected, but wind power appears to affect prices stronger than solar power does. Fuel prices also rank highly and show nontrivial dependencies, including strong interactions with other features revealed by a SHAP interaction analysis. Large generation ramps are correlated with high prices, again with strong feature interactions, due to the limited flexibility of nuclear and lignite plants. Our results further contribute to model development by providing quantitative insights directly from data.Comment: 13 pages, 6 figure

Multivariate scenario generation of day-ahead electricity prices using normalizing flows

Trading on the day-ahead electricity markets requires accurate information about the realization of electricity prices and the uncertainty attached to the predictions. Deriving accurate forecasting models presents a difficult task due to the day-ahead price’s non-stationarity resulting from changing market conditions, e.g., due to changes resulting from the energy crisis in 2021. We present a probabilistic forecasting approach for day-ahead electricity prices using the fully data-driven deep generative model called normalizing flow. Our modeling approach generates full-day scenarios of day-ahead electricity prices based on conditional features such as residual load forecasts. Furthermore, we propose extended feature sets of prior realizations and a periodic retraining scheme that allows the normalizing flow to adapt to the changing conditions of modern electricity markets. Our results highlight that the normalizing flow generates high-quality scenarios that reproduce the true price distribution and yield accurate forecasts. Additionally, our analysis highlights how our improvements towards adaptations in changing regimes allow the normalizing flow to adapt to changing market conditions and enable continued sampling of high-quality day-ahead price scenarios.Multivariate scenario generation of day-ahead electricity prices using normalizing flowspublishedVersio