Exploring auction based energy trade with the support of MAS and blockchain technology

This document describes a simulation of the local energy market with support of multi-agent approach and blockchain technology. The investigated points include blockchain technology and its applications, Ethereum platform and smart contracts as a tool for storing data of operations and creating assets, multi-agent approach to model the local energy market. The document explores building a solution for proposed problem with blockchain technology, agent interactions on the simulated market and auction models, that provide sustainability and profit for the local energy market overall

New actor types in electricity market simulation models: Deliverable D4.4

Project TradeRES - New Markets Design & Models for 100% Renewable Power Systems: https://traderes.eu/about/ABSTRACT: The modelling of agents in the simulation models and tools is of primary importance if the quality and the validity of the simulation outcomes are at stake. This is the first version of the report that deals with the representation of electricity market actors’ in the agent based models (ABMs) used in TradeRES project. With the AMIRIS, the EMLab-Generation (EMLab), the MASCEM and the RESTrade models being in the centre of the analysis, the subject matter of this report has been the identification of the actors’ characteristics that are already covered by the initial (with respect to the project) version of the models and the presentation of the foreseen modelling enhancements. For serving these goals, agent attributes and representation methods, as found in the literature of agent-driven models, are considered initially. The detailed review of such aspects offers the necessary background and supports the formation of a context that facilitates the mapping of actors’ characteristics to agent modelling approaches. Emphasis is given in several approaches and technics found in the literature for the development of a broader environment, on which part of the later analysis is deployed. Although the ABMs that are used in the project constitute an important part of the literature, they have not been included in the review since they are the subject of another section.N/

A novel soft computing approach based on FIR to model and predict energy dynamic systems

Tesi en modalitat compendi de publicacionsWe are facing a global climate crisis that is demanding a change in the status quo of how we produce, distribute and consume energy. In the last decades, this is being redefined through Smart Grids(SG), an intelligent electrical network more observable, controllable, automated, fully integrated with energy services and the end-users. Most of the features and proposed SG scenarios are based on reliable, robust and fast energy predictions. For instance, for proper planning activities, such as generation, purchasing, maintenance and investment; for demand side management, like demand response programs; for energy trading, especially at local level, where productions and consumptions are more stochastics and dynamic; better forecasts also increase grid stability and thus supply security. A large variety of Artificial Intelligence(AI) techniques have been applied in the field of Short-term electricity Load Forecasting(SLF) at consumer level in low-voltage system, showing a better performance than classical techniques. Inaccuracy or failure in the SLF process may be translated not just in a non-optimal (low prediction accuracy) solution but also in frustration of end-users, especially in new services and functionalities that empower citizens. In this regard, some limitations have been observed in energy forecasting models based on AI such as robustness, reliability, accuracy and computation in the edge. This research proposes and develops a new version of Fuzzy Inductive Reasoning(FIR), called Flexible FIR, to model and predict the electricity consumption of an entity in the low-voltage grid with high uncertainties, and information missing, as well as the capacity to be deployed either in the cloud or locally in a new version of Smart Meters(SMs) based on Edge Computing(EC). FIR has been proved to be a powerful approach for model identification and system ’s prediction over dynamic and complex processes in different real world domains but not yet in the energy domain. Thus, the main goal of this thesis is to demonstrate that a new version of FIR, more robust, reliable and accurate can be a referent Soft Computing(SC) methodology to model and predict dynamic systems in the energy domain and that it is scalable to an EC integration. The core developments of Flexible FIR have been an algorithm that can cope with missing information in the input values, as well as learn from instances with Missing Values(MVs) in the knowledge-based, without compromising significantly the accuracy of the predictions. Moreover, Flexible FIR comes with new forecasting strategies that can cope better with loss of causality of a variable and dispersion of output classes than classical k nearest neighbours, making the FIR forecasting process more reliable and robust. Furthermore, Flexible FIR addresses another major challenge modelling with SC techniques, which is to select best model parameters. One of the most important parameters in FIR is the number k of nearest neighbours to be used in the forecast process. The challenge to select the optimal k, dynamically, is addressed through an algorithm, called KOS(K nearest neighbour Optimal Selection), which has been developed and tested also with real world data. It computes a membership aggregation function of all the neighbours with respect their belonging to the output classes.While with KOS the optimal parameter k is found online, with other approaches such as genetic algorithms or reinforcement learning is not, which increases the computational time.Ens trobem davant una crisis climàtica global que exigeix un canvi al status quo de la manera que produïm, distribuïm i consumim energia. En les darreres dècades, està sent redefinit gràcies a les xarxa elèctriques intel·ligents(SG: Smart Grid) amb millor observabilitat, control, automatització, integrades amb nous serveis energètics i usuaris finals. La majoria de les funcionalitats i escenaris de les SG es basen en prediccions de la càrrega elèctrica confiables, robustes i ràpides. Per les prediccions de càrregues elèctriques a curt termini(SLF: Short-term electricity Load Forecasting), a nivell de consumidors al baix voltatge, s’han aplicat una gran varietat de tècniques intel·ligència Artificial(IA) mostrant millor rendiment que tècniques estadístiques tradicionals. Un baix rendiment en SLF, pot traduir-se no només en una solució no-òptima (baixa precisió de predicció) sinó també en la frustració dels usuaris finals, especialment en nous serveis i funcionalitats que empoderarien als ciutadans. En el marc d’aquesta investigació es proposa i desenvolupa una nova versió de la metodologia del Raonament Inductiu Difús(FIR: Fuzzy Inductive Reasoning), anomenat Flexible FIR, capaç de modelar i predir el consum d’electricitat d’una entitat amb un grau d’incertesa molt elevat, inclús amb importants carències d’informació (missing values). A més, Flexible FIR té la capacitat de desplegar-se al núvol, així como localment, en el que podria ser una nova versió de Smart Meters (SM) basada en tecnologia d’Edge Computing (EC). FIR ja ha demostrat ser una metodologia molt potent per la generació de models i prediccions en processos dinàmics en diferents àmbits, però encara no en el de l’energia. Per tant, l’objectiu principal d’aquesta tesis és demostrar que una versió millorada de FIR, més robusta, fiable i precisa pot consolidar-se com una metodologia Soft Computing SC) de referencia per modelar i predir sistemes dinàmics en aplicacions per al sector de l’energia i que és escalable a una integració d’EC. Les principals millores de Flexible FIR han estat, en primer lloc, el desenvolupament i test d’un algorisme capaç de processar els valors d’entrada d’un model FIR tot i que continguin Missing Values (MV). Addicionalment, aquest algorisme també permet aprendre d’instàncies amb MV en la matriu de coneixement d’un model FIR, sense comprometre de manera significativa la precisió de les prediccions. En segon lloc, s’han desenvolupat i testat noves estratègies per a la fase de predicció, comportant-se millor que els clàssics k veïns més propers quan ens trobem amb pèrdua de causalitat d’una variable i dispersió en les classes de sortida, aconseguint un procés d’aprenentatge i predicció més confiable i robust. En tercer lloc, Flexible FIR aborda un repte molt comú en tècniques de SC: l’òptima parametrització del model. En FIR, un dels paràmetres més determinants és el número k de veïns més propers que s’utilitzaran durant la fase de predicció. La selecció del millor valor de k es planteja de manera dinàmica a través de l’algorisme KOS (K nearest neighbour Optimal Selection) que s’ha desenvolupat i testat també amb dades reals. Mentre que amb KOS el paràmetre òptim de k es calcula online, altres enfocaments mitjançant algoritmes genètics o aprenentatge per reforç el càlcul és offline, incrementant significativament el temps de resposta, sent a més a més difícil la implantació en escenaris d’EC. Aquestes millores fan que Flexible FIR es pugui adaptar molt bé en aplicacions d’EC. En aquest sentit es proposa el concepte d’un SM de segona generació basat en EC, que integra Flexible FIR com mòdul de predicció d’electricitat executant-se en el propi dispositiu i un agent EC amb capacitat per el trading d'energia produïda localment. Aquest agent executa un innovador mecanisme basat en incentius, anomenat NRG-X-Change que utilitza una nova moneda digital descentralitzada per l’intercanvi d’energia, que s’anomena NRGcoin.Estamos ante una crisis climática global que exige un cambio del status quo de la manera que producimos, distribuimos y consumimos energía. En las últimas décadas, este status quo está siendo redefinido debido a: la penetración de las energías renovables y la generación distribuida; nuevas tecnologías como baterías y paneles solares con altos rendimientos; y la forma en que se consume la energía, por ejemplo, a través de vehículos eléctricos o con la electrificación de los hogares. Estas palancas requieren una red eléctrica inteligente (SG: Smart Grid) con mayor observabilidad, control, automatización y que esté totalmente integrada con nuevos servicios energéticos, así como con sus usuarios finales. La mayoría de las funcionalidades y escenarios de las redes eléctricas inteligentes se basan en predicciones de la energía confiables, robustas y rápidas. Por ejemplo, para actividades de planificación como la generación, compra, mantenimiento e inversión; para la gestión de la demanda, como los programas de demand response; en el trading de electricidad, especialmente a nivel local, donde las producciones y los consumos son más estocásticos y dinámicos; una mejor predicción eléctrica también aumenta la estabilidad de la red y, por lo tanto, mejora la seguridad. Para las predicciones eléctricas a corto plazo (SLF: Short-term electricity Load Forecasting), a nivel de consumidores en el bajo voltaje, se han aplicado una gran variedad de técnicas de Inteligencia Artificial (IA) mostrando mejor rendimiento que técnicas estadísticas convencionales. Un bajo rendimiento en los modelos predictivos, puede traducirse no solamente en una solución no-óptima (baja precisión de predicción) sino también en frustración de los usuarios finales, especialmente en nuevos servicios y funcionalidades que empoderan a los ciudadanos. En este sentido, se han identificado limitaciones en modelos de predicción de energía basados en IA, como la robustez, fiabilidad, precisión i computación en el borde. En el marco de esta investigación se propone y desarrolla una nueva versión de la metodología de Razonamiento Inductivo Difuso (FIR: Fuzzy Inductive Reasoning), que hemos llamado Flexible FIR, capaz de modelar y predecir el consumo de electricidad de una entidad con altos grados de incertidumbre e incluso con importantes carencias de información (missing values). Además, Flexible FIR tiene la capacidad de desplegarse en la nube, así como localmente, en lo que podría ser una nueva versión de Smart Meters (SM) basada en tecnología de Edge Computing (EC). En el pasado, ya se ha demostrado que FIR es una metodología muy potente para la generación de modelos y predicciones en procesos dinámicos, sin embargo, todavía no ha sido demostrado en el campo de la energía. Por tanto, el objetivo principal de esta tesis es demostrar que una versión mejorada de FIR, más robusta, fiable y precisa puede consolidarse como metodología Soft Computing (SC) de referencia para modelar y predecir sistemas dinámicos en aplicaciones para el sector de la energía y que es escalable hacia una integración de EC. Las principales mejoras en Flexible FIR han sido, en primer lugar, el desarrollo y testeo de un algoritmo capaz de procesar los valores de entrada en un modelo FIR a pesar de que contengan Missing Values (MV). Además, dicho algoritmo también permite aprender de instancias con MV en la matriz de conocimiento de un modelo FIR, sin comprometer de manera significativa la precisión de las predicciones. En segundo lugar, se han desarrollado y testeado nuevas estrategias para la fase de predicción de un modelo FIR, comportándose mejor que los clásicos k vecinos más cercanos ante la pérdida de causalidad de una variable y dispersión de clases de salida, consiguiendo un proceso de aprendizaje y predicción más confiable y robusto. En tercer lugar, Flexible FIR aborda un desafío muy común en técnicas de SC: la óptima parametrización del modelo. En FIR, uno de los parámetros más determinantes es el número k de vecinos más cercanos que se utilizarán en la fase de predicción. La selección del mejor valor de k se plantea de manera dinámica a través del algoritmo KOS (K nearest neighbour Optimal Selection) que se ha desarrollado y probado también con datos reales. Dicho algoritmo calcula una función de membresía agregada, de todos los vecinos, con respecto a su pertenencia a las clases de salida. Mientras que con KOS el parámetro óptimo de k se calcula online, otros enfoques mediante algoritmos genéticos o aprendizaje por refuerzo, el cálculo es offline incrementando significativamente el tiempo de respuesta, siendo además difícil su implantación en escenarios de EC. Estas mejoras hacen que Flexible FIR se adapte muy bien en aplicaciones de EC, en las que la analítica de datos en streaming debe ser fiable, robusta y con un modelo suficientemente ligero para ser ejecutado en un IoT Gateway o dispositivos más pequeños. También, en escenarios con poca conectividad donde el uso de la computación en la nube es limitado y los parámetros del modelo se calculan localmente. Con estas premisas, en esta tesis, se propone el concepto de un SM de segunda generación basado en EC, que integra Flexible FIR como módulo de predicción de electricidad ejecutándose en el dispositivo y un agente EC con capacidad para el trading de energía producida localmente. Dicho agente ejecuta un novedoso mecanismo basado en incentivos, llamado NRG-X-Change que utiliza una nueva moneda digital descentralizada para el intercambio de energía, llamada NRGcoin.Postprint (published version

Optimal energy management for a grid-tied solar PV-battery microgrid: A reinforcement learning approach

There has been a shift towards energy sustainability in recent years, and this shift should continue. The steady growth of energy demand because of population growth, as well as heightened worries about the number of anthropogenic gases released into the atmosphere and deployment of advanced grid technologies, has spurred the penetration of renewable energy resources (RERs) at different locations and scales in the power grid. As a result, the energy system is moving away from the centralized paradigm of large, controllable power plants and toward a decentralized network based on renewables. Microgrids, either grid-connected or islanded, provide a key solution for integrating RERs, load demand flexibility, and energy storage systems within this framework. Nonetheless, renewable energy resources, such as solar and wind energy, can be extremely stochastic as they are weather dependent. These resources coupled with load demand uncertainties lead to random variations on both the generation and load sides, thus challenging optimal energy management. This thesis develops an optimal energy management system (EMS) for a grid-tied solar PV-battery microgrid. The goal of the EMS is to obtain the minimum operational costs (cost of power exchange with the utility and battery wear cost) while still considering network constraints, which ensure grid violations are avoided. A reinforcement learning (RL) approach is proposed to minimize the operational cost of the microgrid under this stochastic setting. RL is a reward-motivated optimization technique derived from how animals learn to optimize their behaviour in new environments. Unlike other conventional model-based optimization approaches, RL doesn't need an explicit model of the optimization system to get optimal solutions. The EMS is modelled as a Markov Decision Process (MDP) to achieve optimality considering the state, action, and reward function. The feasibility of two RL algorithms, namely, conventional Q-learning algorithm and deep Q network algorithm, are developed, and their efficacy in performing optimal energy management for the designed system is evaluated in this thesis. First, the energy management problem is expressed as a sequential decision-making process, after which two algorithms, trading, and non-trading algorithm, are developed. In the trading algorithm case, excess microgrid's energy can be sold back to the utility to increase revenue, while in the latter case constraining rules are embedded in the designed EMS to ensure that no excess energy is sold back to the utility. Then a Q-learning algorithm is developed to minimize the operational cost of the microgrid under unknown future information. Finally, to evaluate the performance of the proposed EMS, a comparison study between a trading case EMS model and a non-trading case is performed using a typical commercial load curve and PV generation profile over a 24- hour horizon. Numerical simulation results indicated that the algorithm learned to select an optimized energy schedule that minimizes energy cost (cost of power purchased from the utility based on the time-varying tariff and battery wear cost) in both summer and winter case studies. However, comparing the non-trading EMS to the trading EMS model operational costs, the latter one decreased cost by 4.033% in the summer season and 2.199% in the winter season. Secondly, a deep Q network (DQN) method that uses recent learning algorithm enhancements, including experience replay and target network, is developed to learn the system uncertainties, including load demand, grid prices and volatile power supply from the renewables solve the optimal energy management problem. Unlike the Q-learning method, which updates the Q-function using a lookup table (which limits its scalability and overall performance in stochastic optimization), the DQN method uses a deep neural network that approximates the Q- function via statistical regression. The performance of the proposed method is evaluated with differently fluctuating load profiles, i.e., slow, medium, and fast. Simulation results substantiated the efficacy of the proposed method as the algorithm was established to learn from experience to raise the battery state of charge and optimally shift loads from a one-time instance, thus supporting the utility grid in reducing aggregate peak load. Furthermore, the performance of the proposed DQN approach was compared to the conventional Q-learning algorithm in terms of achieving a minimum global cost. Simulation results showed that the DQN algorithm outperformed the conventional Q-learning approach, reducing system operational costs by 15%, 24%, and 26% for the slow, medium, and fast fluctuating load profiles in the studied cases

Understanding Deregulated Retail Electricity Markets in the Future: A Perspective from Machine Learning and Optimization

On top of Smart Grid technologies and new market mechanism design, the further deregulation of retail electricity market at distribution level will play a important role in promoting energy system transformation in a socioeconomic way. In today’s retail electricity market, customers have very limited ”energy choice,” or freedom to choose different types of energy services. Although the installation of distributed energy resources (DERs) has become prevalent in many regions, most customers and prosumers who have local energy generation and possible surplus can still only choose to trade with utility companies.They either purchase energy from or sell energy surplus back to the utilities directly while suffering from some price gap. The key to providing more energy trading freedom and open innovation in the retail electricity market is to develop new consumer-centric business models and possibly a localized energy trading platform. This dissertation is exactly pursuing these ideas and proposing a holistic localized electricity retail market to push the next-generation retail electricity market infrastructure to be a level playing field, where all customers have an equal opportunity to actively participate directly. This dissertation also studied and discussed opportunities of many emerging technologies, such as reinforcement learning and deep reinforcement learning, for intelligent energy system operation. Some improvement suggestion of the modeling framework and methodology are included as well.Ph.D.College of Engineering & Computer ScienceUniversity of Michigan-Dearbornhttps://deepblue.lib.umich.edu/bitstream/2027.42/145686/1/Tao Chen Final Dissertation.pdfDescription of Tao Chen Final Dissertation.pdf : Dissertatio

Peer-to-Peer Energy Trading in Smart Residential Environment with User Behavioral Modeling

Electric power systems are transforming from a centralized unidirectional market to a decentralized open market. With this shift, the end-users have the possibility to actively participate in local energy exchanges, with or without the involvement of the main grid. Rapidly reducing prices for Renewable Energy Technologies (RETs), supported by their ease of installation and operation, with the facilitation of Electric Vehicles (EV) and Smart Grid (SG) technologies to make bidirectional flow of energy possible, has contributed to this changing landscape in the distribution side of the traditional power grid. Trading energy among users in a decentralized fashion has been referred to as Peer- to-Peer (P2P) Energy Trading, which has attracted significant attention from the research and industry communities in recent times. However, previous research has mostly focused on engineering aspects of P2P energy trading systems, often neglecting the central role of users in such systems. P2P trading mechanisms require active participation from users to decide factors such as selling prices, storing versus trading energy, and selection of energy sources among others. The complexity of these tasks, paired with the limited cognitive and time capabilities of human users, can result sub-optimal decisions or even abandonment of such systems if performance is not satisfactory. Therefore, it is of paramount importance for P2P energy trading systems to incorporate user behavioral modeling that captures users’ individual trading behaviors, preferences, and perceived utility in a realistic and accurate manner. Often, such user behavioral models are not known a priori in real-world settings, and therefore need to be learned online as the P2P system is operating. In this thesis, we design novel algorithms for P2P energy trading. By exploiting a variety of statistical, algorithmic, machine learning, and behavioral economics tools, we propose solutions that are able to jointly optimize the system performance while taking into account and learning realistic model of user behavior. The results in this dissertation has been published in IEEE Transactions on Green Communications and Networking 2021, Proceedings of IEEE Global Communication Conference 2022, Proceedings of IEEE Conference on Pervasive Computing and Communications 2023 and ACM Transactions on Evolutionary Learning and Optimization 2023