Bilateral contract prices estimation using a Q-learning based approach

The electricity markets restructuring process encouraged the use of computational tools in order to allow the study of different market mechanisms and the relationships between the participating entities. Automated negotiation plays a crucial role in the decision support for energy transactions due to the constant need for players to engage in bilateral negotiations. This paper proposes a methodology to estimate bilateral contract prices, which is essential to support market players in their decisions, enabling adequate risk management of the negotiation process. The proposed approach uses an adaptation of the Q-Learning reinforcement learning algorithm to choose the best from a set of possible contract prices forecasts that are determined using several methods, such as artificial neural networks (ANN), support vector machines (SVM), among others. The learning process assesses the probability of success of each forecasting method, by comparing the expected negotiation price with the historic data contracts of competitor players. The negotiation scenario identified as the most probable scenario that the player will face during the negotiation process is the one that presents the higher expected utility value. This approach allows the supported player to be prepared for the negotiation scenario that is the most likely to represent a reliable approximation of the actual negotiation environment.This work has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 703689 (project ADAPT) and No 641794 (project DREAM-GO); NetEfficity Project (P2020 − 18015); and UID/EEA/00760/2013 funded by FEDER Funds through COMPETE pro-gram and by National Funds through FCT.info:eu-repo/semantics/publishedVersio

Towards Integration of Cognitive Models in Dialogue Management: Designing the Virtual Negotiation Coach Application

This paper presents an approach to flexible and adaptive dialogue management driven by cognitive modelling of human dialogue behaviour. Artificial intelligent agents, based on the ACT-R cognitive architecture, together with human actors are participating in a (meta)cognitive skills training within a negotiation scenario. The agent  employs instance-based learning to decide about its own actions and to reflect on the behaviour of the opponent. We show that task-related actions can be handled by a cognitive agent who is a plausible dialogue partner.  Separating task-related and dialogue control actions enables the application of sophisticated models along with a flexible architecture  in which  various alternative modelling methods can be combined. We evaluated the proposed approach with users assessing  the relative contribution of various factors to the overall usability of a dialogue system. Subjective perception of effectiveness, efficiency and satisfaction were correlated with various objective performance metrics, e.g. number of (in)appropriate system responses, recovery strategies, and interaction pace. It was observed that the dialogue system usability is determined most by the quality of agreements reached in terms of estimated Pareto optimality, by the user's negotiation strategies selected, and by the quality of system recognition, interpretation and responses. We compared human-human and human-agent performance with respect to the number and quality of agreements reached, estimated cooperativeness level, and frequency of accepted negative outcomes. Evaluation experiments showed promising, consistently positive results throughout the range of the relevant scales

An automatic tool to Extract, Transform and Load data from real electricity markets

The study of Electricity Markets operation has been gaining an increasing importance in the last years, as result of the new challenges that the restructuring produced. Currently, lots of information concerning Electricity Markets is available, as market operators provide, after a period of confidentiality, data regarding market proposals and transactions. These data can be used as source of knowledge, to define realistic scenarios, essential for understanding and forecast Electricity Markets behaviour. The development of tools able to extract, transform, store and dynamically update data, is of great importance to go a step further into the comprehension of Electricity Markets and the behaviour of the involved entities. In this paper we present an adaptable tool capable of downloading, parsing and storing data from market operators’ websites, assuring actualization and reliability of stored data

NEGOSEIO: framework for the sustainability of model-oriented enterprise interoperability

Dissertation to obtain the degree of Doctor of Philosophy in Electrical and Computer Engineering(Industrial Information Systems)This dissertation tackles the problematic of Enterprise Interoperability in the current globally connected world. The evolution of the Information and Communication Technologies has endorsed the establishment of fast, secure and robust data exchanges, promoting the development of networked solutions. This allowed the specialisation of enterprises (particularly SMEs) and favoured the development of complex and heterogeneous provider systems. Enterprises are abandoning their self-centrism and working together on the development of more complete solutions. Entire business solutions are built integrating several enterprises (e.g., in supply chains, enterprise nesting) towards a common objective. Additionally, technologies, platforms, trends, standards and regulations keep evolving and demanding enterprises compliance. This evolution needs to be continuous, and is naturally followed by a constant update of each networked enterprise’s interfaces, assets, methods and processes. This unstable environment of perpetual change is causing major concerns in both SMEs and customers as the current interoperability grounds are frail, easily leading to periods of downtime, where business is not possible. The pressure to restore interoperability rapidly often leads to patching and to the adoption of immature solutions, contributing to deteriorate even more the interoperable environment. This dissertation proposes the adoption of NEGOSEIO, a framework that tackles interoperability issues by developing strong model-based knowledge assets and promoting continuous improvement and adaptation for increasing the sustainability of interoperability on enterprise systems. It presents the research motivations and the developed framework’s main blocks, which include model-based knowledge management, collaboration service-oriented architectures implemented over a cloud-based solution, and focusing particularly on its negotiation core mechanism to handle inconsistencies and solutions for the detected interoperability problems. It concludes by validating the research and the proposed framework, presenting its application in a real business case of aerospace mission design on the European Space Agency (ESA).FP7 ENSEMBLE, UNITE, MSEE and IMAGINE project

Context aware Q-Learning-based model for decision support in the negotiation of energy contracts

[EN] Automated negotiation plays a crucial role in the decision support for bilateral energy transactions. In fact, an adequate analysis of past actions of opposing negotiators can improve the decision-making process of market players, allowing them to choose the most appropriate parties to negotiate with in order to increase their outcomes. This paper proposes a new model to estimate the expected prices that can be achieved in bilateral contracts under a specific context, enabling adequate risk management in the negotiation process. The proposed approach is based on an adaptation of the Q-Learning reinforcement learning algorithm to choose the best scenario (set of forecast contract prices) from a set of possible scenarios that are determined using several forecasting and estimation methods. The learning process assesses the probability of occurrence of each scenario, by comparing each expected scenario with the real scenario. The final chosen scenario is the one that presents the higher expected utility value. Besides, the learning method can determine which is the best scenario for each context, since the behaviour of players can change according to the negotiation environment. Consequently, these conditions influence the final contract price of negotiations. This approach allows the supported player to be prepared for the negotiation scenario that is the most probable to represent a reliable approximation of the actual negotiation environme

Artificial Intelligence and Machine Learning Approaches to Energy Demand-Side Response: A Systematic Review

Recent years have seen an increasing interest in Demand Response (DR) as a means to provide flexibility, and hence improve the reliability of energy systems in a cost-effective way. Yet, the high complexity of the tasks associated with DR, combined with their use of large-scale data and the frequent need for near real-time de-cisions, means that Artificial Intelligence (AI) and Machine Learning (ML) — a branch of AI — have recently emerged as key technologies for enabling demand-side response. AI methods can be used to tackle various challenges, ranging from selecting the optimal set of consumers to respond, learning their attributes and pref-erences, dynamic pricing, scheduling and control of devices, learning how to incentivise participants in the DR schemes and how to reward them in a fair and economically efficient way. This work provides an overview of AI methods utilised for DR applications, based on a systematic review of over 160 papers, 40 companies and commercial initiatives, and 21 large-scale projects. The papers are classified with regards to both the AI/ML algorithm(s) used and the application area in energy DR. Next, commercial initiatives are presented (including both start-ups and established companies) and large-scale innovation projects, where AI methods have been used for energy DR. The paper concludes with a discussion of advantages and potential limitations of reviewed AI techniques for different DR tasks, and outlines directions for future research in this fast-growing area

Tasks for Agent-Based Negotiation Teams:Analysis, Review, and Challenges

An agent-based negotiation team is a group of interdependent agents that join together as a single negotiation party due to their shared interests in the negotiation at hand. The reasons to employ an agent-based negotiation team may vary: (i) more computation and parallelization capabilities, (ii) unite agents with different expertise and skills whose joint work makes it possible to tackle complex negotiation domains, (iii) the necessity to represent different stakeholders or different preferences in the same party (e.g., organizations, countries, and married couple). The topic of agent-based negotiation teams has been recently introduced in multi-agent research. Therefore, it is necessary to identify good practices, challenges, and related research that may help in advancing the state-of-the-art in agent-based negotiation teams. For that reason, in this article we review the tasks to be carried out by agent-based negotiation teams. Each task is analyzed and related with current advances in different research areas. The analysis aims to identify special challenges that may arise due to the particularities of agent-based negotiation teams.Comment: Engineering Applications of Artificial Intelligence, 201