Decision Support System for Opponents Selection in Electricity Markets Bilateral Negotiations

This paper presents a new multi-agent decision support system with the purpose of aiding bilateral contract negotiators in the pre-negotiation phase, through the analysis of their possible opponents. The application area of this system is the electricity market, in which players trade a certain volume of energy at a specified price. Consequently, the main output of this system is the recommendation of the best opponent(s) to trade with and the target energy volume to trade with each of the opponents. These recommendations are achieved through the analysis of the possible opponents’ past behavior, namely by learning on their past actions. The result is the forecasting of the expected prices against each opponent depending on the volume to trade. The expected prices are then used by a game-theory based model, to reach the final decision on the best opponents to negotiate with and the ideal target volume to be negotiated with each of themThis work has been developed under the MAS-SOCIETY project - PTDC/EEI-EEE/28954/2017 and has received funding from UID/EEA/00760/2019, funded by FEDER Funds through COMPETE and by National Funds through FCTinfo:eu-repo/semantics/publishedVersio

Identifying Most Probable Negotiation Scenario in Bilateral Contracts with Reinforcement Learning

This paper proposes an adaptation of the Q-Learning reinforcement learning algorithm, for the identification of the most probable scenario that a player may face, under different contexts, when negotiating bilateral contracts. For that purpose, the proposed methodology is integrated in a Decision Support System that is capable to generate several different scenarios for each negotiation context. With this complement, the tool can also identify the most probable scenario for the identified negotiation context. A realistic case study is conducted, based on real contracts data, which confirms the learning capabilities of the proposed methodology. It is possible to identify the most probable scenario for each context over the learned period. Nonetheless, the identified scenario might not always be the real negotiation scenario, given the variable nature of such negotiations. However, this work greatly reduces the frequency of such unexpected scenarios, contributing to a greater success of the supported player over time.This work has received funding from National Funds through FCT (Fundaçao da Ciencia e Tecnologia) under the project SPET – 29165, call SAICT 2017.info:eu-repo/semantics/publishedVersio

Optimizing Opponents Selection in Bilateral Contracts Negotiation with Particle Swarm

This paper proposes a model based on particle swarm optimization to aid electricity markets players in the selection of the best player(s) to trade with, to maximize their bilateral contracts outcome. This approach is integrated in a Decision Support System (DSS) for the pre-negotiation of bilateral contracts, which provides a missing feature in the state-of-art, the possible opponents analysis. The DSS determines the best action of all the actions that the supported player can take, by applying a game theory approach. However, the analysis of all actions can easily become very time-consuming in large negotiation scenarios. The proposed approach aims to provide the DSS with an alternative method with the capability of reducing the execution time while keeping the results quality as much as possible. Both approaches are tested in a realistic case study where the supported player could take almost half a million different actions. The results show that the proposed methodology is able to provide optimal and near-optimal solutions with an huge execution time reduction.This work has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 641794 (project DREAM-GO) and grant agreement No 703689 (project ADAPT); from the CONTEST project - SAICT-POL/23575/2016; and from FEDER Funds through COMPETE program and from National Funds through FCT under the project UID/EEA/00760/2013.info:eu-repo/semantics/publishedVersio

A Multi-Agent System Simulation Model for Trusted Local Energy Markets

The energy market and electric grid play a major role in everyday life. Most areas in modern society, such as: communication, health, transportation, the financial system and many others; require electrical energy to operate properly. Traditionally energy grids operate in a centralized manner. Consumers are connected to centralized utilities in the grid and energy flows from producers to Consumers. However, the rising in popularity in Renewable Energy Sources (RES) such as photovoltaic panels installed in households, small commerce and small industry wide spread the use of distributed energy generation, which the main energy grid was not designed to support. One of the possible solutions for this problem is the creation of a Local Energy Market (LeM). A LeM is a market that operates in a small physical area such as a neighborhood. Traditional consumers can become active market participants under a LeM. That is possible because the LeM is structured in such a way as to enable small-scale negotiations and energy exchanges between participants, who traditionally would only be final consumers. The LeM is capable of dealing with distributed energy generation from RES because negotiations and distribution happen at a local level, thus reducing problems with the main grid. Furthermore, the participation in the local market can reduce energy costs or even create profits for consumers, while contributing to easy the management of the grid and associated technical losses. This work explores the concept of LeM and is focused on two main objectives: designing and developing a system that allows the simulation of LeM, and designing and developing a mechanism that allows trusted negotiations in this market. To accomplish these objectives a Multi-Agent System (MAS) architecture is proposed to model and allow the simulation of LeM. Furthermore to support the market it is also proposed a trust model used to evaluate the behavior of participants and detecting faulty or malicious activities. The developed MAS models a LeM based on a Smart Grid, that is an energy grid with a cyber-physical system with smart meters and communications mechanisms. The MAS was developed with agents to model sensors, market participants and a Market Interaction Manager (MIM) agent that is responsible for managing the negotiations and for applying trust mechanisms. The trust mechanism was designed to attribute a dynamic trust value to each participant, which is reviewed during the all negotiation period. This evaluation of the participant’s trust is based on the analysis of historical data, contextual data, such as weather conditions, and by using forecasting methods to predict the participant expected behavior, allowing to penalize the ones that are exhibiting a questionable behavior in the market. A case study simulation was made with the objective of understanding how the proposed trust mechanism performed, and how the use of different forecasting methods can interfere with it. The results obtained allowed us to conclude that the trust methodology is able to update the trust of each participant, during the negotiation period, and when paired with a well performing forecasting mechanism it is able to achieve a trusted evaluation of the participants behavior. Taking into consideration these results we believe that the proposed trust methodology is capable of providing a valuable trust assessment when used by the MIM agent. This Master Thesis is developed within the scope of a project called Secure interactions and trusted Participation in local Electricity Trading (SPET), a FCT-SAICT2017 funded Research & Development project. SPET project envisions the development of a MAS that is designed to model and simulate the operations of a LeM, taking a focus on security and market trust necessary in this negotiation environment.O mercado de energia e a rede elétrica desempenham um papel importante na vida quotidiana da população. Grande parte das áreas da sociedade moderna, como é o caso da comunicação, transportes, saúde, sistema financeiro, entre outras; requer energia elétrica para funcionar corretamente. Tradicionalmente, as redes de energia operam de forma centralizada. Os consumidores estão conectados a fornecedores centralizados na rede e a energia é transferida dos produtores para os consumidores. No entanto, o aumento da popularidade das Fontes de Energia Renováveis (FER), como painéis fotovoltaicos instalados nas residências, pequeno comércio e pequena indústria, difundiu o uso da geração distribuída de energia, que a rede principal de energia não foi projetada para suportar. Uma das possíveis soluções para esse problema é a criação de um Mercado Local de Energia (MLe). Um MLe é um mercado que opera numa pequena área física, como uma vizinhança. Num MLe, os consumidores tradicionais têm a possibilidade de ser participantes ativos no mercado. Isto é possível porque o MLe está estruturado de forma a permitir negociações em pequena escala e trocas de energia entre os participantes, que tradicionalmente seriam apenas consumidores finais. O MLe é capaz de lidar com a geração de energia distribuída proveniente das FER, porque as negociações e a distribuição ocorrem a um nível local, reduzindo assim os problemas com a rede principal. Para além disso, a participação no mercado local pode reduzir os custos de energia ou até gerar lucros para os consumidores, contribuindo ainda para facilitar a gestão da rede e reduzir as perdas técnicas a ela associadas. Este trabalho explora o conceito de MLe e está focado em dois objetivos principais: projetar e desenvolver um sistema que permita a simulação de MLe, bem como um mecanismo que permita negociações confiáveis neste mercado. Para atingir estes objetivos, é proposta uma arquitetura de Sistema Multi-Agente (SMA) para modelar e permitir a simulação do MLe. Para além disso, para apoiar o mercado, também é proposto um modelo de confiança utilizado para avaliar o comportamento dos participantes e detetar falhas ou atividades maliciosas. O SMA desenvolvido modela um MLe com base numa Smart Grid, que é uma rede de energia com um sistema ciber-físico, com sensores inteligentes e mecanismos de comunicação. O SMA foi desenvolvido com agentes para modelar sensores, participantes do mercado e um agente Market Interaction Manager (MIM), responsável pela gestão das negociações e pela aplicação de mecanismos de confiança. O mecanismo de confiança foi projetado para atribuir um valor de confiança dinâmico a cada participante, que é adaptado durante todo o período de negociação. Essa avaliação da confiança do participante é baseada na análise de dados históricos, contextuais, como condições climatéricas, e no uso de métodos de previsão para antever o comportamento esperado do participante, permitindo penalizar aqueles que exibem um comportamento questionável no mercado. Foi realizada uma simulação de caso de estudo, com o objetivo de avaliar o desempenho do mecanismo de confiança proposto e de que forma é que o uso de diferentes métodos de previsão interfere neste desempenho. Os resultados obtidos permitiram concluir que a metodologia de confiança é capaz de atualizar a confiança de cada participante, durante o período de negociação e, quando combinada com um mecanismo de previsão com bom desempenho, é capaz de obter uma avaliação confiável do comportamento dos participantes. Tendo em consideração estes resultados, acreditamos que a metodologia de confiança proposta é capaz de fornecer uma avaliação de confiança valiosa quando usada pelo agente MIM. Esta tese de mestrado é desenvolvida no âmbito de um projeto chamado Secure interactions and trusted Participation in local Electricity Trading (SPET), um projeto de Investigação e Desenvolvimento (I&D) financiado pela FCT-SAICT2017. O projeto SPET tem como objetivo o desenvolvimento de um MAS para a modelação e simulação de MLe, tendo como foco a segurança e confiança necessárias neste ambiente de negociação

Lightweight Blockchain Framework for Location-aware Peer-to-Peer Energy Trading

Peer-to-Peer (P2P) energy trading can facilitate integration of a large number of small-scale producers and consumers into energy markets. Decentralized management of these new market participants is challenging in terms of market settlement, participant reputation and consideration of grid constraints. This paper proposes a blockchain-enabled framework for P2P energy trading among producer and consumer agents in a smart grid. A fully decentralized market settlement mechanism is designed, which does not rely on a centralized entity to settle the market and encourages producers and consumers to negotiate on energy trading with their nearby agents truthfully. To this end, the electrical distance of agents is considered in the pricing mechanism to encourage agents to trade with their neighboring agents. In addition, a reputation factor is considered for each agent, reflecting its past performance in delivering the committed energy. Before starting the negotiation, agents select their trading partners based on their preferences over the reputation and proximity of the trading partners. An Anonymous Proof of Location (A-PoL) algorithm is proposed that allows agents to prove their location without revealing their real identity. The practicality of the proposed framework is illustrated through several case studies, and its security and privacy are analyzed in detail

Context aware Q-Learning-based model for decision support in the negotiation of energy contracts

[EN] Automated negotiation plays a crucial role in the decision support for bilateral energy transactions. In fact, an adequate analysis of past actions of opposing negotiators can improve the decision-making process of market players, allowing them to choose the most appropriate parties to negotiate with in order to increase their outcomes. This paper proposes a new model to estimate the expected prices that can be achieved in bilateral contracts under a specific context, enabling adequate risk management in the negotiation process. The proposed approach is based on an adaptation of the Q-Learning reinforcement learning algorithm to choose the best scenario (set of forecast contract prices) from a set of possible scenarios that are determined using several forecasting and estimation methods. The learning process assesses the probability of occurrence of each scenario, by comparing each expected scenario with the real scenario. The final chosen scenario is the one that presents the higher expected utility value. Besides, the learning method can determine which is the best scenario for each context, since the behaviour of players can change according to the negotiation environment. Consequently, these conditions influence the final contract price of negotiations. This approach allows the supported player to be prepared for the negotiation scenario that is the most probable to represent a reliable approximation of the actual negotiation environme

Tinbergen and Theil Meet Nash: Controllability in Policy Games

This paper generalizes the classical theory of economic policy to a static LQ-strategic context between n players. We show how this generalized version of controllability can profitably be used to deal with policy ineffectiveness issues and Nash equilibrium existence.Policy games, policy ineffectiveness, static controllability, Nash equilibrium existence

A Multi-Agent Energy Trading Competition

The energy sector will undergo fundamental changes over the next ten years. Prices for fossil energy resources are continuously increasing, there is an urgent need to reduce CO2 emissions, and the United States and European Union are strongly motivated to become more independent from foreign energy imports. These factors will lead to installation of large numbers of distributed renewable energy generators, which are often intermittent in nature. This trend conflicts with the current power grid control infrastructure and strategies, where a few centralized control centers manage a limited number of large power plants such that their output meets the energy demands in real time. As the proportion of distributed and intermittent generation capacity increases, this task becomes much harder, especially as the local and regional distribution grids where renewable energy generators are usually installed are currently virtually unmanaged, lack real time metering and are not built to cope with power flow inversions (yet). All this is about to change, and so the control strategies must be adapted accordingly. While the hierarchical command-and-control approach served well in a world with a few large scale generation facilities and many small consumers, a more flexible, decentralized, and self-organizing control infrastructure will have to be developed that can be actively managed to balance both the large grid as a whole, as well as the many lower voltage sub-grids. We propose a competitive simulation test bed to stimulate research and development of electronic agents that help manage these tasks. Participants in the competition will develop intelligent agents that are responsible to level energy supply from generators with energy demand from consumers. The competition is designed to closely model reality by bootstrapping the simulation environment with real historic load, generation, and weather data. The simulation environment will provide a low-risk platform that combines simulated markets and real-world data to develop solutions that can be applied to help building the self-organizing intelligent energy grid of the future

Blockchain and internet of things for electrical energy decentralization: A review and system architecture

Decentralization in electrical power grids has gained increasing importance, especially in the last two decades, since transmission system operators (TSO), distribution system operators (DSO) and consumers are more aware of energy efficiency and energy sustainability issues. Therefore, globally, due to the introduction of energy production technologies near the consumers, in residential and industrial sectors, new scenarios of decentralized energy production (DEP) are emerging. To guarantee an adequate power management in the electrical power grids, incorporating producers, consumers, and producers-consumers, commonly designated as prosumers together, it is important to adopt intelligent systems and platforms that allow the provision of information on energy consumption and production in real time, as well as for obtaining the price for the sale and purchase of energy. In this research the literature is analysed to identify the appropriate solutions to implement a decentralized electrical power grid based on sensors, blockchain and smart contracts, evaluating the current state of the art and pilot projects already in place. A conceptual model for a power grid model is presented, with renewable energy production, combining Internet of Things (IoT), blockchain and smart contracts.A descentralização nas redes elétricas ganhou importância crescente, especialmente nas últimas duas décadas, uma vez que os operadores da rede de transporte (ORT), operadores da rede de distribuição (ORD) e consumidores estão mais conscientes das questões de eficiência energética e sustentabilidade energética. Globalmente, devido à introdução de tecnologias de produção de energia junto dos consumidores, nos setores residencial e industrial, estão a surgir novos cenários de produção de energia descentralizada. Para garantir uma adequada gestão de energia nas redes elétricas, integrando produtores, consumidores e produtores-consumidores, vulgarmente designados por prosumers, é importante adotar sistemas e plataformas inteligentes que permitam fornecer informações sobre consumo e produção de energia em tempo real, bem como para obter o preço de compra e venda de energia. Nesta pesquisa, a literatura é analisada para identificar as soluções adequadas para implementar uma rede elétrica descentralizada baseada em sensores, blockchain e contratos inteligentes, avaliando o estado da arte atual e projetos piloto já em curso. É apresentado um modelo conceptual para um modelo de rede elétrica, com produção de energia renovável, combinando Internet das Coisas (IoT), blockchain e contratos inteligentes

Smart Grid Enabling Low Carbon Future Power Systems Towards Prosumers Era

In efforts to meet the targets of carbon emissions reduction in power systems, policy makers formulate measures for facilitating the integration of renewable energy sources and demand side carbon mitigation. Smart grid provides an opportunity for bidirectional communication among policy makers, generators and consumers. With the help of smart meters, increasing number of consumers is able to produce, store, and consume energy, giving them the new role of prosumers. This thesis aims to address how smart grid enables prosumers to be appropriately integrated into energy markets for decarbonising power systems. This thesis firstly proposes a Stackelberg game-theoretic model for dynamic negotiation of policy measures and determining optimal power profiles of generators and consumers in day-ahead market. Simulation results show that the proposed model is capable of saving electricity bills, reducing carbon emissions, and increasing the penetration of renewable energy sources. Secondly, a data-driven prosumer-centric energy scheduling tool is developed by using learning approaches to reduce computational complexity from model-based optimisation. This scheduling tool exploits convolutional neural networks to extract prosumption patterns, and uses scenarios to analyse possible variations of uncertainties caused by the intermittency of renewable energy sources and flexible demand. Case studies confirm that the proposed scheduling tool can accurately predict optimal scheduling decisions under various system scales and uncertain scenarios. Thirdly, a blockchain-based peer-to-peer trading framework is designed to trade energy and carbon allowance. The bidding/selling prices of individual prosumers can directly incentivise the reshaping of prosumption behaviours. Case studies demonstrate the execution of smart contract on the Ethereum blockchain and testify that the proposed trading framework outperforms the centralised trading and aggregator-based trading in terms of regional energy balance and reducing carbon emissions caused by long-distance transmissions