A Projective Simulation Scheme for Partially-Observable Multi-Agent Systems

We introduce a kind of partial observability to the projective simulation (PS) learning method. It is done by adding a belief projection operator and an observability parameter to the original framework of the efficiency of the PS model. I provide theoretical formulations, network representations, and situated scenarios derived from the invasion toy problem as a starting point for some multi-agent PS models.Comment: 28 pages, 21 figure

A Multiagent Q-learning-based Restoration Algorithm for Resilient Distribution System Operation

Natural disasters, human errors, and technical issues have caused disastrous blackouts to power systems and resulted in enormous economic losses. Moreover, distributed energy resources have been integrated into distribution systems, which bring extra uncertainty and challenges to system restoration. Therefore, the restoration of power distribution systems requires more efficient and effective methods to provide resilient operation. In the literature, using Q-learning and multiagent system (MAS) to restore power systems has the limitation in real system application, without considering power system operation constraints. In order to adapt to system condition changes quickly, a restoration algorithm using Q-learning and MAS, together with the combination method and battery algorithm is proposed in this study. The developed algorithm considers voltage and current constraints while finding system switching configuration to maximize the load pick-up after faults happen to the given system. The algorithm consists of three parts. First, it finds switching configurations using Q-learning. Second, the combination algorithm works as a back-up plan in case of the solution from Q-learning violates system constraints. Third, the battery algorithm is applied to determine the charging or discharging schedule of battery systems. The obtained switching configuration provides restoration solutions without violating system constraints. Furthermore, the algorithm can adjust switching configurations after the restoration. For example, when renewable output changes, the algorithm provides an adjusted solution to avoid violating system constraints. The proposed algorithm has been tested in the modified IEEE 9-bus system using the real-time digital simulator. Simulation results demonstrate that the algorithm offers an efficient and effective restoration strategy for resilient distribution system operation

Multiagent Deep Reinforcement Learning: Challenges and Directions Towards Human-Like Approaches

This paper surveys the field of multiagent deep reinforcement learning. The combination of deep neural networks with reinforcement learning has gained increased traction in recent years and is slowly shifting the focus from single-agent to multiagent environments. Dealing with multiple agents is inherently more complex as (a) the future rewards depend on the joint actions of multiple players and (b) the computational complexity of functions increases. We present the most common multiagent problem representations and their main challenges, and identify five research areas that address one or more of these challenges: centralised training and decentralised execution, opponent modelling, communication, efficient coordination, and reward shaping. We find that many computational studies rely on unrealistic assumptions or are not generalisable to other settings; they struggle to overcome the curse of dimensionality or nonstationarity. Approaches from psychology and sociology capture promising relevant behaviours such as communication and coordination. We suggest that, for multiagent reinforcement learning to be successful, future research addresses these challenges with an interdisciplinary approach to open up new possibilities for more human-oriented solutions in multiagent reinforcement learning.Comment: 37 pages, 6 figure