Emergence of Scenario-Appropriate Collaborative Behaviors for Teams of Robotic Bodyguards

We are considering the problem of controlling a team of robotic bodyguards protecting a VIP from physical assault in the presence of neutral and/or adversarial bystanders. This task is part of a much larger class of problems involving coordinated robot behavior in the presence of humans. This problem is challenging due to the large number of active entities with different agendas, the need of cooperation between the robots as well as the requirement to take into consideration criteria such as social norms and unobtrusiveness in addition to the main goal of VIP safety. Furthermore, different settings such as street, public space or red carpet require very different behavior from the robot. We describe how a multi-agent reinforcement learning approach can evolve behavior policies for teams of robot bodyguards that compare well with hand-engineered approaches. Furthermore, we show that an algorithm inspired by universal value function approximators can learn policies that exhibit appropriate, distinct behavior in environments with different requirements.Comment: Accepted for publication at AAMAS 201

Survey on Unmanned Aerial Vehicle Networks: A Cyber Physical System Perspective

Unmanned aerial vehicle (UAV) networks are playing an important role in various areas due to their agility and versatility, which have attracted significant attention from both the academia and industry in recent years. As an integration of the embedded systems with communication devices, computation capabilities and control modules, the UAV network could build a closed loop from data perceiving, information exchanging, decision making to the final execution, which tightly integrates the cyber processes into the physical devices. Therefore, the UAV network could be considered as a cyber physical system (CPS). Revealing the coupling effects among the three interacted components in this CPS system, i.e., communication, computation and control, is envisioned as the key to properly utilize all the available resources and hence improve the performance of the UAV networks. In this paper, we present a comprehensive survey on the UAV networks from a CPS perspective. Firstly, we respectively research the basics and advances with respect to the three CPS components in the UAV networks. Then we look inside to investigate how these components contribute to the system performance by classifying the UAV networks into three hierarchies, i.e., the cell level, the system level, and the system of system level. Further, the coupling effects among these CPS components are explicitly illustrated, which could be enlightening to deal with the challenges in each individual aspect. New research directions and open issues are discussed at the end of this survey. With this intensive literature review, we try to provide a novel insight into the state-of-the-art in the UAV networks.Comment: 42 pages, 19 figures and 11 table