Multistep Electricity Price Forecasting for Deregulated Energy Markets: GAN-Based Reinforcement Learning

Electricity Price Forecasting (EPF) plays a vital role in smart grid applications for deregulated electricity markets. Most of the studies tend to investigate the electricity market influencers using forecasting techniques, often losing sight of significance on the sensibility of EPF models to the unstable real-time environment. This project will address a novel EPF based on deep reinforcement learning. The proposed approach uses generative adversarial networks (GAN) to collect synthetic data and increase training set effectively and increase the adaptation of the forecasting system to the environment. The data collected will be fed to a Deep Q learning to generate the final predictions. The proposed GAN-DQL will also be assessed on real data to prove the proposed model advantages compared to several machine learning solutions

Assessing the role of human behaviors in the management of extreme hydrological events: an agent-based modeling approach

This thesis aims to assess the role of human behaviors in the management of extreme hydrological events. Using an agent-based modeling (ABM) approach, three specific issues associated with modeling human behaviors are addressed: (1) behavioral heterogeneity, (2) social interaction, and (3) the interplay of multiple behaviors. The modeling approach is applied to two types of extreme hydrological events: floods and droughts. In the case of flood events, an ABM is developed to simulate heterogeneous responses to flood warnings and evacuation decisions. The ABM is coupled with a traffic model to simulate evacuation processes on a transportation network in an impending flood event. Based on this coupled framework, the model further takes account of social interactions, in the form of communication through social media, and evaluates how social interactions affect flood risk awareness and evacuation processes. The case of drought events considers a hypothetical agricultural water market based on double auction. Farmers’ multiple behaviors (irrigation and bidding behaviors) are modeled in an ABM framework. The impacts of the interplay of these behaviors on water market performance are evaluated under various hydrological conditions. The results from the ABMs show that the three aforementioned aspects of human behaviors can significantly affect the effectiveness of the management policies in extreme hydrological events. The thesis highlights the importance of including human behaviors for policy design in flood and drought management. Further, the thesis emphasizes the efforts in collecting empirical data to better represent and simulate human behaviors in coupled human and hydrological systems

Essays on finance, learning, and macroeconomics

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Economics, 2012.Cataloged from PDF version of thesis.Includes bibliographical references (p. 189-198).This thesis consists of four essays on finance, learning, and macroeconomics. The first essay studies whether learning can explain why the standard consumption-based asset pricing model produces large pricing errors for U.S. equity returns. I prove that under learning standard moment conditions need not hold in finite samples, leading to pricing errors. Simulations show that learning can generate quantitatively realistic pricing errors and a substantial equity risk premium. I find that a model with learning is not rejected in the data, producing pricing errors that are statistically indistinguishable from zero. The second essay (co-authored with Anna Mikusheva) studies the properties of the common impulse response function matching estimator (IRFME) in settings with many parameters. We prove that the common IRFME is consistent and asymptotically normal only when the horizon of IRFs being matched grows slowly enough. We use simulations to evaluate the performance of the common IRFME in a practical example, and we compare it with an infrequently used bias corrected approach, based on indirect inferences. Our findings suggest that the common IRFME performs poorly in situations where the sample size is not much larger than the horizon of IRFs being matched, and in those situations, the bias corrected approach with bootstrapped standard errors performs better. The third essay (co-authored with Ricardo Caballero) documents that, in contrast with their widely perceived excess return, popular carry trade strategies yield low systemicrisk- adjusted returns. In contrast, hedging the carry with exchange rate options produces large returns that are not a compensation for systemic risk. We show that this result stems from the fact that the corresponding portfolio of exchange rate options provides a cheap form of systemic insurance. The fourth essay shows that the documented overbidding in pay-as-you-go auctions relative to a static model can be explained by the presence of a small subset of aggressive bidders. I argue that aggressive bidding can be rational if users are able to form reputations that deter future competition, and I present empirical evidence that this is the case. In auctions without any aggressive bidders, there is no evidence of overbidding in PAYGA.by Joseph Buchman Doyle, Jr.Ph.D

Adaptive interaction protocol for multi-agent-based supply network

Supply network is a special form of organization, where two paradigms are combined: market interaction and hierarchical structure. Because it includes different interdependent entities that should work together, supply network requires efficient tools to support coordination and improve the quality of interaction between these entities. In this thesis, we propose a multi-agent-based interaction protocol for supply network formation. The main characteristic of this protocol is its adaptability to agent behavior during supply network formation. Such an adaptability is needed to form the most suitable organization structure. The novelty of the proposed protocol is the unification of contracting, auction and negotiation in a three-steps solution. Contracting is a planning tool aiming to facilitate solving network formation problem through task decomposition. Auction enables participating agents to jointly search an agreement space and check the contract feasibility. Finally, negotiation is the last resort for agents to reach an agreement. It is applicable when agents fail to reach an agreement due to lack of knowledge about the existing constraints. In fact, negotiation is used to release constraints when no realizable solution is obtained under auction. The second contribution of this thesis is the consideration of supply network formation within a general framework of coordinated distributed problem solving. The proposed protocol and framework are simulated through a multi-agent system prototype serving as a proof of concept. The simulation results show the effectiveness of our three-steps solution protocol in terms of network formation success, customer satisfaction and the total gained rewards of the whole network

The impact of timber price uncertainty and flexible harvest on bidding behavior in N +1 price forest biodiversity auctions

Conservation auctions have received increased attention since the turn of the century. Improved management of forests for biodiversity is one of the areas where this has been the case. Previous studies have to my knowledge not accounted for the impact of timber price variability on forest owners’ bidding behavior. This thesis fills this void in the literature. Brazee and Mendelsohn (1988) showed that timber price fluctuations affect the timing of timber cutting as the forest owners' reservation prices for cutting increase under higher price variability. A basic insight from auction theory is that no bidder will submit a bid that makes him or her worse off. This is also the case for conservation contract auctions. I combine the above two insights to show that bids for forest conservation auctions will become higher under timber price variability compared to cases with timber prices without such fluctuations.M-ECO

Detecting and Forecasting Economic Regimes in Multi-Agent Automated Exchanges

We show how an autonomous agent can use observable market conditions to characterize the microeconomic situation of the market and predict future market trends. The agent can use this information to make both tactical decisions, such as pricing, and strategic decisions, such as product mix and production planning. We develop methods to learn dominant market conditions, such as over-supply or scarcity, from historical data using Gaussian mixture models to construct price density functions. We discuss how this model can be combined with real-time observable information to identify the current dominant market condition and to forecast market changes over a planning horizon. We forecast market changes via both a Markov correction-prediction process and an exponential smoother. Empirical analysis shows that the exponential smoother yields more accurate predictions for the current and the next day (supporting tactical decisions), while the Markov correction-prediction process is better for longer term predictions (supporting strategic decisions). Our approach offers more flexibility than traditional regression based approaches, since it does not assume a fixed functional relationship between dependent and independent variables. We validate our methods by presenting experimental results in a case study, the Trading Agent Competition for Supply Chain Management.dynamic pricing;machine learning;market forecasting;Trading agents

A novel multi-level and community-based agent ecosystem to support customers dynamic decision-making in smart grids

Electrical systems have evolved at a fast pace over the past years, particularly in response to the current environmental and climate challenges. Consequently, the European Union and the United Nations have encouraged the development of a more sustainable energy strategy. This strategy triggered a paradigm shift in energy consumption and production, which becoming increasingly distributed, resulted in the development and emergence of smart energy grids. Multi-agent systems are one of the most widely used artificial intelligence concepts in smart grids. Both multi-agent systems and smart grids are distributed, so there is correspondence between the used technology and the network's complex reality. Due to the wide variety of multi-agent systems applied to smart grids, which typically have very specific goals, the ability to model the network as a whole may be compromised, as communication between systems is typically non-existent. This dissertation, therefore, proposes an agent-based ecosystem to model smart grids in which different agent-based systems can coexist. This dissertation aims to conceive, implement, test, and validate a new agent-based ecosystem, entitled A4SG (agent-based ecosystem for smart grids modelling), which combines the concepts of multi-agent systems and agent communities to enable the modelling and representation of smart grids and the entities that compose them. The proposed ecosystem employs an innovative methodology for managing static or dynamic interactions present in smart grids. The creation of a solution that allows the integration of existing systems into an ecosystem, enables the representation of smart grids in a realistic and comprehensive manner. A4SG integrates several functionalities that support the ecosystem's management, also conceived, implemented, tested, and validated in this dissertation. Two mobility functionalities are proposed: one that allows agents to move between physical machines and another that allows "virtual" mobility, where agents move between agent communities to improve the context for the achievement of their objectives. In order to prevent an agent from becoming overloaded, a novel functionality is proposed to enable the creation of agents that function as extensions of the main agent (i.e., branch agents), allowing the distribution of objectives among the various extensions of the main agent. Several case studies, which test the proposed services and functionalities individually and the ecosystem as a whole, were used to test and validate the proposed solution. These case studies were conducted in realistic contexts using data from multiple sources, including energy communities. The results indicate that the used methodologies can increase participation in demand response events, increasing the fitting between consumers and aggregators from 12 % to 69 %, and improve the strategies used in energy transaction markets, allowing an energy community of 50 customers to save 77.0 EUR per week.Os últimos anos têm sido de mudança nos sistemas elétricos, especialmente devido aos atuais desafios ambientais e climáticos. A procura por uma estratégia mais sustentável para o domínio da energia tem sido promovida pela União Europeia e pela Organização das Nações Unidas. A mudança de paradigma no que toca ao consumo e produção de energia, que acontece, cada vez mais, de forma distribuída, tem levado à emergência das redes elétricas inteligentes. Os sistemas multi-agente são um dos conceitos, no domínio da inteligência artificial, mais aplicados em redes inteligentes. Tanto os sistemas multi-agente como as redes inteligentes têm uma natureza distribuída, existindo por isso um alinhamento entre a tecnologia usada e a realidade complexa da rede. Devido a existir uma vasta oferta de sistemas multi-agente aplicados a redes inteligentes, normalmente com objetivos bastante específicos, a capacidade de modelar a rede como um todo pode ficar comprometida, porque a comunicação entre sistemas é, geralmente, inexistente. Por isso, esta dissertação propõe um ecossistema baseado em agentes para modelar as redes inteligentes, onde vários sistemas de agentes coexistem. Esta dissertação pretende conceber, implementar, testar, e validar um novo ecossistema multiagente, intitulado A4SG (agent-based ecosystem for smart grids modelling), que combina os conceitos de sistemas multi-agente e comunidades de agentes, permitindo a modelação e representação de redes inteligentes e das suas entidades. O ecossistema proposto utiliza uma metodologia inovadora para gerir as interações presentes nas redes inteligentes, sejam elas estáticas ou dinâmicas. A criação de um ecossistema que permite a integração de sistemas já existentes, cria a possibilidade de uma representação realista e detalhada das redes de energia. O A4SG integra diversas funcionalidades, também estas concebidas, implementadas, testadas, e validadas nesta dissertação, que suportam a gestão do próprio ecossistema. São propostas duas funcionalidades de mobilidade, uma que permite aos agentes mover-se entre máquinas físicas, e uma que permite uma mobilidade “virtual”, onde os agentes se movem entre comunidades de agentes, de forma a melhorar o contexto para a execução dos seus objetivos. É também proposta uma nova funcionalidade que permite a criação de agentes que funcionam como uma extensão de um agente principal, com o objetivo de evitar a sobrecarga de um agente, permitindo a distribuição de objetivos entre as várias extensões do agente principal. A solução proposta foi testada e validada por vários casos de estudo, que testam os serviços e funcionalidades propostas individualmente, e o ecossistema como um todo. Estes casos de estudo foram executados em contextos realistas, usando dados provenientes de diversas fontes, tais como comunidades de energia. Os resultados demonstram que as metodologias utilizadas podem melhorar a participação em eventos de demand response, subindo a adequação entre consumidores e agregadores de 12 % para 69 %, e melhorar as estratégias utilizadas em mercados de transações de energia, permitindo a uma comunidade de energia com 50 consumidores poupar 77,0 EUR por semana

What to bid and when to stop

Negotiation is an important activity in human society, and is studied by various disciplines, ranging from economics and game theory, to electronic commerce, social psychology, and artificial intelligence. Traditionally, negotiation is a necessary, but also time-consuming and expensive activity. Therefore, in the last decades there has been a large interest in the automation of negotiation, for example in the setting of e-commerce. This interest is fueled by the promise of automated agents eventually being able to negotiate on behalf of human negotiators.Every year, automated negotiation agents are improving in various ways, and there is now a large body of negotiation strategies available, all with their unique strengths and weaknesses. For example, some agents are able to predict the opponent's preferences very well, while others focus more on having a sophisticated bidding strategy. The problem however, is that there is little incremental improvement in agent design, as the agents are tested in varying negotiation settings, using a diverse set of performance measures. This makes it very difficult to meaningfully compare the agents, let alone their underlying techniques. As a result, we lack a reliable way to pinpoint the most effective components in a negotiating agent.There are two major advantages of distinguishing between the different components of a negotiating agent's strategy: first, it allows the study of the behavior and performance of the components in isolation. For example, it becomes possible to compare the preference learning component of all agents, and to identify the best among them. Second, we can proceed to mix and match different components to create new negotiation strategies., e.g.: replacing the preference learning technique of an agent and then examining whether this makes a difference. Such a procedure enables us to combine the individual components to systematically explore the space of possible negotiation strategies.To develop a compositional approach to evaluate and combine the components, we identify structure in most agent designs by introducing the BOA architecture, in which we can develop and integrate the different components of a negotiating agent. We identify three main components of a general negotiation strategy; namely a bidding strategy (B), possibly an opponent model (O), and an acceptance strategy (A). The bidding strategy considers what concessions it deems appropriate given its own preferences, and takes the opponent into account by using an opponent model. The acceptance strategy decides whether offers proposed by the opponent should be accepted.The BOA architecture is integrated into a generic negotiation environment called Genius, which is a software environment for designing and evaluating negotiation strategies. To explore the negotiation strategy space of the negotiation research community, we amend the Genius repository with various existing agents and scenarios from literature. Additionally, we organize a yearly international negotiation competition (ANAC) to harvest even more strategies and scenarios. ANAC also acts as an evaluation tool for negotiation strategies, and encourages the design of negotiation strategies and scenarios.We re-implement agents from literature and ANAC and decouple them to fit into the BOA architecture without introducing any changes in their behavior. For each of the three components, we manage to find and analyze the best ones for specific cases, as described below. We show that the BOA framework leads to significant improvements in agent design by wining ANAC 2013, which had 19 participating teams from 8 international institutions, with an agent that is designed using the BOA framework and is informed by a preliminary analysis of the different components.In every negotiation, one of the negotiating parties must accept an offer to reach an agreement. Therefore, it is important that a negotiator employs a proficient mechanism to decide under which conditions to accept. When contemplating whether to accept an offer, the agent is faced with the acceptance dilemma: accepting the offer may be suboptimal, as better offers may still be presented before time runs out. On the other hand, accepting too late may prevent an agreement from being reached, resulting in a break off with no gain for either party. We classify and compare state-of-the-art generic acceptance conditions. We propose new acceptance strategies and we demonstrate that they outperform the other conditions. We also provide insight into why some conditions work better than others and investigate correlations between the properties of the negotiation scenario and the efficacy of acceptance conditions.Later, we adopt a more principled approach by applying optimal stopping theory to calculate the optimal decision on the acceptance of an offer. We approach the decision of whether to accept as a sequential decision problem, by modeling the bids received as a stochastic process. We determine the optimal acceptance policies for particular opponent classes and we present an approach to estimate the expected range of offers when the type of opponent is unknown. We show that the proposed approach is able to find the optimal time to accept, and improves upon all existing acceptance strategies.Another principal component of a negotiating agent's strategy is its ability to take the opponent's preferences into account. The quality of an opponent model can be measured in two different ways. One is to use the agent's performance as a benchmark for the model's quality. We evaluate and compare the performance of a selection of state-of-the-art opponent modeling techniques in negotiation. We provide an overview of the factors influencing the quality of a model and we analyze how the performance of opponent models depends on the negotiation setting. We identify a class of simple and surprisingly effective opponent modeling techniques that did not receive much previous attention in literature.The other way to measure the quality of an opponent model is to directly evaluate its accuracy by using similarity measures. We review all methods to measure the accuracy of an opponent model and we then analyze how changes in accuracy translate into performance differences. Moreover, we pinpoint the best predictors for good performance. This leads to new insights concerning how to construct an opponent model, and what we need to measure when optimizing performance.Finally, we take two different approaches to gain more insight into effective bidding strategies. We present a new classification method for negotiation strategies, based on their pattern of concession making against different kinds of opponents. We apply this technique to classify some well-known negotiating strategies, and we formulate guidelines on how agents should bid in order to be successful, which gives insight into the bidding strategy space of negotiating agents. Furthermore, we apply optimal stopping theory again, this time to find the concessions that maximize utility for the bidder against particular opponents. We show there is an interesting connection between optimal bidding and optimal acceptance strategies, in the sense that they are mirrored versions of each other.Lastly, after analyzing all components separately, we put the pieces back together again. We take all BOA components accumulated so far, including the best ones, and combine them all together to explore the space of negotiation strategies.We compute the contribution of each component to the overall negotiation result, and we study the interaction between components. We find that combining the best agent components indeed makes the strongest agents. This shows that the component-based view of the BOA architecture not only provides a useful basis for developing negotiating agents but also provides a useful analytical tool. By varying the BOA components we are able to demonstrate the contribution of each component to the negotiation result, and thus analyze the significance of each. The bidding strategy is by far the most important to consider, followed by the acceptance conditions and finally followed by the opponent model.Our results validate the analytical approach of the BOA framework to first optimize the individual components, and then to recombine them into a negotiating agent