14,994 research outputs found
On Similarities between Inference in Game Theory and Machine Learning
In this paper, we elucidate the equivalence between inference in game theory and machine learning. Our aim in so doing is to establish an equivalent vocabulary between the two domains so as to facilitate developments at the intersection of both fields, and as proof of the usefulness of this approach, we use recent developments in each field to make useful improvements to the other. More specifically, we consider the analogies between smooth best responses in fictitious play and Bayesian inference methods. Initially, we use these insights to develop and demonstrate an improved algorithm for learning in games based on probabilistic moderation. That is, by integrating over the distribution of opponent strategies (a Bayesian approach within machine learning) rather than taking a simple empirical average (the approach used in standard fictitious play) we derive a novel moderated fictitious play algorithm and show that it is more likely than standard fictitious play to converge to a payoff-dominant but risk-dominated Nash equilibrium in a simple coordination game. Furthermore we consider the converse case, and show how insights from game theory can be used to derive two improved mean field variational learning algorithms. We first show that the standard update rule of mean field variational learning is analogous to a Cournot adjustment within game theory. By analogy with fictitious play, we then suggest an improved update rule, and show that this results in fictitious variational play, an improved mean field variational learning algorithm that exhibits better convergence in highly or strongly connected graphical models. Second, we use a recent advance in fictitious play, namely dynamic fictitious play, to derive a derivative action variational learning algorithm, that exhibits superior convergence properties on a canonical machine learning problem (clustering a mixture distribution)
Resilient Autonomous Control of Distributed Multi-agent Systems in Contested Environments
An autonomous and resilient controller is proposed for leader-follower
multi-agent systems under uncertainties and cyber-physical attacks. The leader
is assumed non-autonomous with a nonzero control input, which allows changing
the team behavior or mission in response to environmental changes. A resilient
learning-based control protocol is presented to find optimal solutions to the
synchronization problem in the presence of attacks and system dynamic
uncertainties. An observer-based distributed H_infinity controller is first
designed to prevent propagating the effects of attacks on sensors and actuators
throughout the network, as well as to attenuate the effect of these attacks on
the compromised agent itself. Non-homogeneous game algebraic Riccati equations
are derived to solve the H_infinity optimal synchronization problem and
off-policy reinforcement learning is utilized to learn their solution without
requiring any knowledge of the agent's dynamics. A trust-confidence based
distributed control protocol is then proposed to mitigate attacks that hijack
the entire node and attacks on communication links. A confidence value is
defined for each agent based solely on its local evidence. The proposed
resilient reinforcement learning algorithm employs the confidence value of each
agent to indicate the trustworthiness of its own information and broadcast it
to its neighbors to put weights on the data they receive from it during and
after learning. If the confidence value of an agent is low, it employs a trust
mechanism to identify compromised agents and remove the data it receives from
them from the learning process. Simulation results are provided to show the
effectiveness of the proposed approach
A Comprehensive Survey of Potential Game Approaches to Wireless Networks
Potential games form a class of non-cooperative games where unilateral
improvement dynamics are guaranteed to converge in many practical cases. The
potential game approach has been applied to a wide range of wireless network
problems, particularly to a variety of channel assignment problems. In this
paper, the properties of potential games are introduced, and games in wireless
networks that have been proven to be potential games are comprehensively
discussed.Comment: 44 pages, 6 figures, to appear in IEICE Transactions on
Communications, vol. E98-B, no. 9, Sept. 201
Continuous Strategy Replicator Dynamics for Multi--Agent Learning
The problem of multi-agent learning and adaptation has attracted a great deal
of attention in recent years. It has been suggested that the dynamics of multi
agent learning can be studied using replicator equations from population
biology. Most existing studies so far have been limited to discrete strategy
spaces with a small number of available actions. In many cases, however, the
choices available to agents are better characterized by continuous spectra.
This paper suggests a generalization of the replicator framework that allows to
study the adaptive dynamics of Q-learning agents with continuous strategy
spaces. Instead of probability vectors, agents strategies are now characterized
by probability measures over continuous variables. As a result, the ordinary
differential equations for the discrete case are replaced by a system of
coupled integral--differential replicator equations that describe the mutual
evolution of individual agent strategies. We derive a set of functional
equations describing the steady state of the replicator dynamics, examine their
solutions for several two-player games, and confirm our analytical results
using simulations.Comment: 12 pages, 15 figures, accepted for publication in JAAMA
- ā¦