Coordination in Networks Formation: Experimental Evidence on Learning and Salience

We present experiments on repeated non-cooperative network formation games, based on Bala and Goyal (2000). We treat the one-way and the two-ways flow models, each for high and low link costs. The models show both multiple equilibria and coordination problems. We conduct experiments under various conditions which control for salient labeling and learning dynamics. Contrary to previous experiments, we find that coordination on non-empty Strict Nash equilibria is not an easy task for subjects to achieve, even in the mono-directional model where the Strict Nash equilibria is a wheel. We find that salience significantly helps coordination, but only when subjects are pre-instructed to think of the wheel network as a reasonable way to play the networking game. Evidence on learning behavior provides support for subjects choosing strategies consistent with various learning rules, which include as the main ones Reinforcement and Fictitious Play.Experiments, Networks, Behavioral game theory, Salience, Learning dynamics

Applications of Agent-Based Methods in Multi-Energy Systems—A Systematic Literature Review

The need for a greener and more sustainable energy system evokes a need for more extensive energy system transition research. The penetration of distributed energy resources and Internet of Things technologies facilitate energy system transition towards the next generation of energy system concepts. The next generation of energy system concepts include “integrated energy system”, “multi-energy system”, or “smart energy system”. These concepts reveal that future energy systems can integrate multiple energy carriers with autonomous intelligent decision making. There are noticeable trends in using the agent-based method in research of energy systems, including multi-energy system transition simulation with agent-based modeling (ABM) and multi-energy system management with multi-agent system (MAS) modeling. The need for a comprehensive review of the applications of the agent-based method motivates this review article. Thus, this article aims to systematically review the ABM and MAS applications in multi-energy systems with publications from 2007 to the end of 2021. The articles were sorted into MAS and ABM applications based on the details of agent implementations. MAS application papers in building energy systems, district energy systems, and regional energy systems are reviewed with regard to energy carriers, agent control architecture, optimization algorithms, and agent development environments. ABM application papers in behavior simulation and policy-making are reviewed with regard to the agent decision-making details and model objectives. In addition, the potential future research directions in reinforcement learning implementation and agent control synchronization are highlighted. The review shows that the agent-based method has great potential to contribute to energy transition studies with its plug-and-play ability and distributed decision-making process

How does regulation affect innovation and technology change in the water sector in England and Wales?

This thesis examines the role of regulation in technological change in the water sector in England and Wales. Based on a combination of Social-Ecological Systems (SES) theory and the Multi-Level Perspective on technological transitions a Comparative Information-Graded Approach (CIGA) is developed in Part 1. As part of the CIGA, a series of tools is used for characterizing and evaluating the relationship between regulation and technology. In Part 2, the CIGA is applied to characterize the relationship between regulation and water innovation in England and Wales based on official publications, Environment Agency data, and interviews. In particular, 7 mechanisms are identified by which regulation affects innovation and 5 issues of trust negatively interact with innovation. As trust is established through these mechanisms, opportunities for innovation are at times sacrificed. Part 3 develops and analyses a set of models based on findings in Part 2. Dynamical systems and fictitious play analysis of a trustee game model of regulation exhibits cyclicality providing an explanation for observed cycles which create an inconsistent drive for innovation. Trustee and coordination models are evaluated in Chapter 7 highlighting how most tools struggle with the issue of technological lock-in. Chapter 8 develops a model of two innovators and a public good water technology over time, showing the role foresight plays in this context as well as the disincentive to develop it. Taken together, the CIGA characterization and modelling work provide a series of recommendations and insights into how the system of regulation affects technology change.Open Acces

A Study of Problems Modelled as Network Equilibrium Flows

This thesis presents an investigation into selfish routing games from three main perspectives. These three areas are tied together by a common thread that runs through the main text of this thesis, namely selfish routing games and network equilibrium flows. First, it investigates methods and models for nonatomic selfish routing and then develops algorithms for solving atomic selfish routing games. A number of algorithms are introduced for the atomic selfish routing problem, including dynamic programming for a parallel network and a metaheuristic tabu search. A piece-wise mixed-integer linear programming problem is also presented which allows standard solvers to solve the atomic selfish routing problem. The connection between the atomic selfish routing problem, mixed-integer linear programming and the multicommodity flow problem is explored when constrained by unsplittable flows or flows that are restricted to a number of paths. Additionally, some novel probabilistic online learning algorithms are presented and compared with the equilibrium solution given by the potential function of the nonatomic selfish routing game. Second, it considers multi-criteria extensions of selfish routing and the inefficiency of the equilibrium solutions when compared with social cost. Models are presented that allow exploration of the Pareto set of solutions for a weighted sum model (akin to the social cost) and the equilibrium solution. A means by which these solutions can be measured based on the Price of Anarchy for selfish routing games is also presented. Third, it considers the importance and criticality of components of the network (edges, vertices or a collection of both) within a selfish routing game and the impact of their removal. Existing network science measures and demand-based measures are analysed to assess the change in total travel time and issues highlighted. A new measure which solves these issues is presented and the need for such a measure is evaluated. Most of the new findings have been disseminated through conference talks and journal articles, while others represent the subject of papers currently in preparation

Optimal decision making in cognitive radio networks

Cognitive Radio Networks are being researched upon heavily in the various layers of the communication structure. The task of bringing software in the physical layer of communication system led to the concept of a smart radio being able to learn, adapt and make intelligent decisions in an autonomous manner by use of a Software Defined Radio. This work provides novel concepts in the areas of spectrum sensing, learning of ongoing transmissions through Reinforcment learning, use of a game theoretic concept such as Zero-sum game for resilience of authorized users in cases of jamming, and decision making of user transmissions through Markov Decision processes. This is highly applicable in dynamic radio environments such as emergency communications required during natural disasters, large scale events and in mobile wireless communications. Such applications come under the "Internet of Things"

Multi-energy retail market simulation with autonomous intelligent agents

Tese de doutoramento. Engenharia Electrotécnica e de Computadores. 2005. Faculdade de Engenharia. Universidade do Port

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field