Solving DCOPs with Distributed Large Neighborhood Search

The field of Distributed Constraint Optimization has gained momentum in recent years, thanks to its ability to address various applications related to multi-agent cooperation. Nevertheless, solving Distributed Constraint Optimization Problems (DCOPs) optimally is NP-hard. Therefore, in large-scale, complex applications, incomplete DCOP algorithms are necessary. Current incomplete DCOP algorithms suffer of one or more of the following limitations: they (a) find local minima without providing quality guarantees; (b) provide loose quality assessment; or (c) are unable to benefit from the structure of the problem, such as domain-dependent knowledge and hard constraints. Therefore, capitalizing on strategies from the centralized constraint solving community, we propose a Distributed Large Neighborhood Search (D-LNS) framework to solve DCOPs. The proposed framework (with its novel repair phase) provides guarantees on solution quality, refining upper and lower bounds during the iterative process, and can exploit domain-dependent structures. Our experimental results show that D-LNS outperforms other incomplete DCOP algorithms on both structured and unstructured problem instances

Multi-objective Decentralised Coordination for Teams of Robotic Agents

This thesis introduces two novel coordination mechanisms for a team of multiple autonomous decision makers, represented as autonomous robotic agents. Such techniques aim to improve the capabilities of robotic agents, such as unmanned aerial or ground vehicles (UAVs and UGVs), when deployed in real world operations. In particular, the work reported in this thesis focuses on improving the decision making of teams of such robotic agents when deployed in an unknown, and dynamically changing, environment to perform search and rescue operations for lost targets. This problem is well known and studied within both academia and industry and coordination mechanisms for controlling such teams have been studied in both the robotics and the multi-agent systems communities. Within this setting, our first contribution aims at solves a canonical target search problem, in which a team of UAVs is deployed in an environment to search for a lost target. Specifically, we present a novel decentralised coordination approach for teams of UAVs, based on the max-sum algorithm. In more detail, we represent each agent as a UAV, and study the applicability of the max-sum algorithm, a decentralised approximate message passing algorithm, to coordinate a team of multiple UAVs for target search. We benchmark our approach against three state-of-the-art approaches within a simulation environment. The results show that coordination with the max-sum algorithm out-performs a best response algorithm, which represents the state of the art in the coordination of UAVs for search, by up to 26%, an implicitly coordinated approach, where the coordination arises from the agents making decisions based on a common belief, by up to 34% and finally a non-coordinated approach by up to 68%. These results indicate that the max-sum algorithm has the potential to be applied in complex systems operating in dynamic environments. We then move on to tackle coordination in which the team has more than one objective to achieve (e.g. maximise the covered space of the search area, whilst minimising the amount of energy consumed by each UAV). To achieve this shortcoming, we present, as our second contribution, an extension of the max-sum algorithm to compute bounded solutions for problems involving multiple objectives. More precisely, we develop the bounded multi-objective max-sum algorithm (B-MOMS), a novel decentralised coordination algorithm able to solve problems involving multiple objectives while providing guarantees on the solution it recovers. B-MOMS extends the standard max-sum algorithm to compute bounded approximate solutions to multi-objective decentralised constraint optimisation problems (MO-DCOPs). Moreover, we prove the optimality of B-MOMS in acyclic constraint graphs, and derive problem dependent bounds on its approximation ratio when these graphs contain cycles. Finally, we empirically evaluate its performance on a multi-objective extension of the canonical graph colouring problem. In so doing, we demonstrate that, for the settings we consider, the approximation ratio never exceeds $2$, and is typically less than $1.5$ for less-constrained graphs. Moreover, the runtime required by B-MOMS on the problem instances we considered never exceeds $30$ minutes, even for maximally constrained graphs with one hundred agents

Solving Transition-Independent Multi-agent MDPs with Sparse Interactions (Extended version)

In cooperative multi-agent sequential decision making under uncertainty, agents must coordinate to find an optimal joint policy that maximises joint value. Typical algorithms exploit additive structure in the value function, but in the fully-observable multi-agent MDP setting (MMDP) such structure is not present. We propose a new optimal solver for transition-independent MMDPs, in which agents can only affect their own state but their reward depends on joint transitions. We represent these dependencies compactly in conditional return graphs (CRGs). Using CRGs the value of a joint policy and the bounds on partially specified joint policies can be efficiently computed. We propose CoRe, a novel branch-and-bound policy search algorithm building on CRGs. CoRe typically requires less runtime than the available alternatives and finds solutions to problems previously unsolvable

Optimal Statistical Rates for Decentralised Non-Parametric Regression with Linear Speed-Up

We analyse the learning performance of Distributed Gradient Descent in the context of multi-agent decentralised non-parametric regression with the square loss function when i.i.d. samples are assigned to agents. We show that if agents hold sufficiently many samples with respect to the network size, then Distributed Gradient Descent achieves optimal statistical rates with a number of iterations that scales, up to a threshold, with the inverse of the spectral gap of the gossip matrix divided by the number of samples owned by each agent raised to a problem-dependent power. The presence of the threshold comes from statistics. It encodes the existence of a "big data" regime where the number of required iterations does not depend on the network topology. In this regime, Distributed Gradient Descent achieves optimal statistical rates with the same order of iterations as gradient descent run with all the samples in the network. Provided the communication delay is sufficiently small, the distributed protocol yields a linear speed-up in runtime compared to the single-machine protocol. This is in contrast to decentralised optimisation algorithms that do not exploit statistics and only yield a linear speed-up in graphs where the spectral gap is bounded away from zero. Our results exploit the statistical concentration of quantities held by agents and shed new light on the interplay between statistics and communication in decentralised methods. Bounds are given in the standard non-parametric setting with source/capacity assumptions

A Survey on Sensor Networks from a Multiagent Perspective

Sensor networks (SNs) have arisen as one of the most promising technologies for the next decades. The recent emergence of small and inexpensive sensors based upon microelectromechanical systems ease the development and proliferation of this kind of networks in a wide range of actual-world applications. Multiagent systems (MAS) have been identified as one of the most suitable technologies to contribute to the deployment of SNs that exhibit flexibility, robustness and autonomy. The purpose of this survey is 2-fold. On the one hand, we review the most relevant contributions of agent technologies to this emerging application domain. On the other hand, we identify the challenges that researchers must address to establish MAS as the key enabling technology for SNs.This work has been funded by projects IEA(TIN2006-15662-C02-01), Agreement Technologies (CONSOLIDER CSD2007-0022, INGENIO 2010), EVE (TIN2009-14702-C02-01,TIN2009-14702-C02-02) and Generalitat de Catalunya under the gran t2009-SGR-1434. Meritxell Vinyals is supported by the Spanish Ministry of Education (FPU grant AP2006-04636)Peer Reviewe

Factored Monte-Carlo tree search for coordinating UAVs in disaster response

The coordination of multiple Unmanned Aerial Vehicles (UAVs) to carry out surveys is a major challenge for emergency responders. In particular, UAVs have to fly over kilometre-scale areas while trying to discover casualties as quickly as possible. However, an increase in the availability of real-time data about a disaster from sources such as crowd reports or satellites presents a valuable source of information to drive the planning of UAV flight paths over a space in order to discover people who are in danger. Nevertheless challenges remain when planning over the very large action spaces that result. To this end, we introduce the survivor discovery problem and present as our solution, the first example of a factored coordinated Monte Carlo tree search algorithm to perform decentralised path planning for multiple coordinated UAVs. Our evaluation against standard benchmarks show that our algorithm, Co-MCTS, is able to find more casualties faster than standard approaches by 10% or more on simulations with real-world data from the 2010 Haiti earthquake