1 research outputs found
A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraint
In teacher-student framework, a more experienced agent (teacher) helps
accelerate the learning of another agent (student) by suggesting actions to
take in certain states. In cooperative multiagent reinforcement learning
(MARL), where agents need to cooperate with one another, a student may fail to
cooperate well with others even by following the teachers' suggested actions,
as the polices of all agents are ever changing before convergence. When the
number of times that agents communicate with one another is limited (i.e.,
there is budget constraint), the advising strategy that uses actions as advices
may not be good enough. We propose a partaker-sharer advising framework (PSAF)
for cooperative MARL agents learning with budget constraint. In PSAF, each
Q-learner can decide when to ask for Q-values and share its Q-values. We
perform experiments in three typical multiagent learning problems. Evaluation
results show that our approach PSAF outperforms existing advising methods under
both unlimited and limited budget, and we give an analysis of the impact of
advising actions and sharing Q-values on agents' learning.Comment: 31 pages, 16 figures, submitted to ACM Transactions on Autonomous and
Adaptive Systems (TAAS