3,272 research outputs found
Learning Multi-Object Positional Relationships via Emergent Communication
The study of emergent communication has been dedicated to interactive
artificial intelligence. While existing work focuses on communication about
single objects or complex image scenes, we argue that communicating
relationships between multiple objects is important in more realistic tasks,
but understudied. In this paper, we try to fill this gap and focus on emergent
communication about positional relationships between two objects. We train
agents in the referential game where observations contain two objects, and find
that generalization is the major problem when the positional relationship is
involved. The key factor affecting the generalization ability of the emergent
language is the input variation between Speaker and Listener, which is realized
by a random image generator in our work. Further, we find that the learned
language can generalize well in a new multi-step MDP task where the positional
relationship describes the goal, and performs better than raw-pixel images as
well as pre-trained image features, verifying the strong generalization ability
of discrete sequences. We also show that language transfer from the referential
game performs better in the new task than learning language directly in this
task, implying the potential benefits of pre-training in referential games. All
in all, our experiments demonstrate the viability and merit of having agents
learn to communicate positional relationships between multiple objects through
emergent communication.Comment: 15 page
- …