Distributed Algorithms for Stochastic Source Seeking With Mobile Robot Networks

Autonomous robot networks are an effective tool for monitoring large-scale environmental fields. This paper proposes distributed control strategies for localizing the source of a noisy signal, which could represent a physical quantity of interest such as magnetic force, heat, radio signal, or chemical concentration. We develop algorithms specific to two scenarios: one in which the sensors have a precise model of the signal formation process and one in which a signal model is not available. In the model-free scenario, a team of sensors is used to follow a stochastic gradient of the signal field. Our approach is distributed, robust to deformations in the group geometry, does not necessitate global localization, and is guaranteed to lead the sensors to a neighborhood of a local maximum of the field. In the model-based scenario, the sensors follow a stochastic gradient of the mutual information (MI) between their expected measurements and the expected source location in a distributed manner. The performance is demonstrated in simulation using a robot sensor network to localize the source of a wireless radio signal

Learning 3D Navigation Protocols on Touch Interfaces with Cooperative Multi-Agent Reinforcement Learning

Using touch devices to navigate in virtual 3D environments such as computer assisted design (CAD) models or geographical information systems (GIS) is inherently difficult for humans, as the 3D operations have to be performed by the user on a 2D touch surface. This ill-posed problem is classically solved with a fixed and handcrafted interaction protocol, which must be learned by the user. We propose to automatically learn a new interaction protocol allowing to map a 2D user input to 3D actions in virtual environments using reinforcement learning (RL). A fundamental problem of RL methods is the vast amount of interactions often required, which are difficult to come by when humans are involved. To overcome this limitation, we make use of two collaborative agents. The first agent models the human by learning to perform the 2D finger trajectories. The second agent acts as the interaction protocol, interpreting and translating to 3D operations the 2D finger trajectories from the first agent. We restrict the learned 2D trajectories to be similar to a training set of collected human gestures by first performing state representation learning, prior to reinforcement learning. This state representation learning is addressed by projecting the gestures into a latent space learned by a variational auto encoder (VAE).Comment: 17 pages, 8 figures. Accepted at The European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases 2019 (ECMLPKDD 2019

Efficient Model Learning for Human-Robot Collaborative Tasks

We present a framework for learning human user models from joint-action demonstrations that enables the robot to compute a robust policy for a collaborative task with a human. The learning takes place completely automatically, without any human intervention. First, we describe the clustering of demonstrated action sequences into different human types using an unsupervised learning algorithm. These demonstrated sequences are also used by the robot to learn a reward function that is representative for each type, through the employment of an inverse reinforcement learning algorithm. The learned model is then used as part of a Mixed Observability Markov Decision Process formulation, wherein the human type is a partially observable variable. With this framework, we can infer, either offline or online, the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this new user and will be robust to deviations of the human actions from prior demonstrations. Finally we validate the approach using data collected in human subject experiments, and conduct proof-of-concept demonstrations in which a person performs a collaborative task with a small industrial robot

Learning to Communicate with Deep Multi-Agent Reinforcement Learning

We consider the problem of multiple agents sensing and acting in environments with the goal of maximising their shared utility. In these environments, agents must learn communication protocols in order to share information that is needed to solve the tasks. By embracing deep neural networks, we are able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability. We propose two approaches for learning in these domains: Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses deep Q-learning, while the latter exploits the fact that, during learning, agents can backpropagate error derivatives through (noisy) communication channels. Hence, this approach uses centralised learning but decentralised execution. Our experiments introduce new environments for studying the learning of communication protocols and present a set of engineering innovations that are essential for success in these domains

Space-Time Hierarchical-Graph Based Cooperative Localization in Wireless Sensor Networks

It has been shown that cooperative localization is capable of improving both the positioning accuracy and coverage in scenarios where the global positioning system (GPS) has a poor performance. However, due to its potentially excessive computational complexity, at the time of writing the application of cooperative localization remains limited in practice. In this paper, we address the efficient cooperative positioning problem in wireless sensor networks. A space-time hierarchical-graph based scheme exhibiting fast convergence is proposed for localizing the agent nodes. In contrast to conventional methods, agent nodes are divided into different layers with the aid of the space-time hierarchical-model and their positions are estimated gradually. In particular, an information propagation rule is conceived upon considering the quality of positional information. According to the rule, the information always propagates from the upper layers to a certain lower layer and the message passing process is further optimized at each layer. Hence, the potential error propagation can be mitigated. Additionally, both position estimation and position broadcasting are carried out by the sensor nodes. Furthermore, a sensor activation mechanism is conceived, which is capable of significantly reducing both the energy consumption and the network traffic overhead incurred by the localization process. The analytical and numerical results provided demonstrate the superiority of our space-time hierarchical-graph based cooperative localization scheme over the benchmarking schemes considered.Comment: 14 pages, 15 figures, 4 tables, accepted to appear on IEEE Transactions on Signal Processing, Sept. 201