A General Learning Framework for Open Ad Hoc Teamwork Using Graph-based Policy Learning

Open ad hoc teamwork is the problem of training a single agent to efficiently collaborate with an unknown group of teammates whose composition may change over time. A variable team composition creates challenges for the agent, such as the requirement to adapt to new team dynamics and dealing with changing state vector sizes. These challenges are aggravated in real-world applications in which the controlled agent only has a partial view of the environment. In this work, we develop a class of solutions for open ad hoc teamwork under full and partial observability. We start by developing a solution for the fully observable case that leverages graph neural network architectures to obtain an optimal policy based on reinforcement learning. We then extend this solution to partially observable scenarios by proposing different methodologies that maintain belief estimates over the latent environment states and team composition. These belief estimates are combined with our solution for the fully observable case to compute an agent's optimal policy under partial observability in open ad hoc teamwork. Empirical results demonstrate that our solution can learn efficient policies in open ad hoc teamwork in fully and partially observable cases. Further analysis demonstrates that our methods' success is a result of effectively learning the effects of teammates' actions while also inferring the inherent state of the environment under partial observability

Establishing Continuous Communication through Dynamic Team Behaviour Switching

Maintaining continuous communication is an important factor that contributes to the success of multi-robot systems. Most research involving multi-robot teams is conducted in controlled laboratory settings, where continuous communication is assumed, typically because there is a wireless network (wifi) that keeps all the robots connected. But for multi-robot teams to operate successfully “in the wild”, it is crucial to consider how communication can be maintained when signals fail or robots move out of range. This paper presents a novel “leader-follower behaviour” with dynamic role switching and messaging that supports uninterrupted communication, regardless of network perturbations. A series of experiments were conducted in which it is shown how network perturbations effect performance, comparing a baseline with the new leaderfollower behaviour. The experiments record metrics on team success, given the two conditions. These results are significant for real-world multi-robot systems applications that require continuous communication amongst team members

Multi-agent Task Allocation for Fruit Picker Team Formation (Extended Abstract)

Multi-agent task allocation methods seek to distribute a set of tasks fairly amongst a set of agents. In real-world settings, such as fruit farms, human labourers undertake harvesting tasks, organised each day by farm manager(s) who assign workers to the fields that are ready to be harvested. The work presented here considers three challenges identified in the adaptation of a multi-agent task allocation methodology applied to the problem of distributing workers to fields. First, the methodology must be fast to compute so that it can be applied on a daily basis. Second, the incremental acquisition of harvesting data used to make decisions about worker-task assignments means that a data-backed approach must be derived from incomplete information as the growing season unfolds. Third, the allocation must take “fairness” into account and consider worker motivation. Solutions to these challenges are demonstrated, showing statistically significant results based on the operations at a soft fruit farm during their 2020 and 2021 harvesting seasons

Information-Theoretic Control of Multiple Sensor Platforms

This thesis is concerned with the development of a consistent, information-theoretic basis for understanding of coordination and cooperation decentralised multi-sensor multi-platform systems. Autonomous systems composed of multiple sensors and multiple platforms potentially have significant importance in applications such as defence, search and rescue mining or intelligent manufacturing. However, the effective use of multiple autonomous systems requires that an understanding be developed of the mechanisms of coordination and cooperation between component systems in pursuit of a common goal. A fundamental, quantitative, understanding of coordination and cooperation between decentralised autonomous systems is the main goal of this thesis. This thesis focuses on the problem of coordination and cooperation for teams of autonomous systems engaged in information gathering and data fusion tasks. While this is a subset of the general cooperative autonomous systems problem, it still encompasses a range of possible applications in picture compilation, navigation, searching and map building problems. The great advantage of restricting the domain of interest in this way is that an underlying mathematical model for coordination and cooperation can be based on the use of information-theoretic models of platform and sensor abilities. The information theoretic approach builds on the established principles and architecture previously developed for decentralised data fusion systems. In the decentralised control problem addressed in this thesis, each platform and sensor system is considered to be a distinct decision maker with an individual information-theoretic utility measure capturing both local objectives and the inter-dependencies among the decisions made by other members of the team. Together these information-theoretic utilities constitute the team objective. The key contributions of this thesis lie in the quantification and study of cooperative control between sensors and platforms using information as a common utility measure. In particular, * The problem of information gathering is formulated as an optimal control problem by identifying formal measures of information with utility or pay-off. * An information-theoretic utility model of coupling and coordination between decentralised decision makers is elucidated. This is used to describe how the information gathering strategies of a team of autonomous systems are coupled. * Static and dynamic information structures for team members are defined. It is shown that the use of static information structures can lead to efficient, although sub-optimal, decentralised control strategies for the team. * Significant examples in decentralised control of a team of sensors are developed. These include the multi-vehicle multi-target bearings-only tracking problem, and the area coverage or exploration problem for multiple vehicles. These examples demonstrate the range of non-trivial problems to which the theory in this thesis can be employed

Information-Theoretic Control of Multiple Sensor Platforms

This thesis is concerned with the development of a consistent, information-theoretic basis for understanding of coordination and cooperation decentralised multi-sensor multi-platform systems. Autonomous systems composed of multiple sensors and multiple platforms potentially have significant importance in applications such as defence, search and rescue mining or intelligent manufacturing. However, the effective use of multiple autonomous systems requires that an understanding be developed of the mechanisms of coordination and cooperation between component systems in pursuit of a common goal. A fundamental, quantitative, understanding of coordination and cooperation between decentralised autonomous systems is the main goal of this thesis. This thesis focuses on the problem of coordination and cooperation for teams of autonomous systems engaged in information gathering and data fusion tasks. While this is a subset of the general cooperative autonomous systems problem, it still encompasses a range of possible applications in picture compilation, navigation, searching and map building problems. The great advantage of restricting the domain of interest in this way is that an underlying mathematical model for coordination and cooperation can be based on the use of information-theoretic models of platform and sensor abilities. The information theoretic approach builds on the established principles and architecture previously developed for decentralised data fusion systems. In the decentralised control problem addressed in this thesis, each platform and sensor system is considered to be a distinct decision maker with an individual information-theoretic utility measure capturing both local objectives and the inter-dependencies among the decisions made by other members of the team. Together these information-theoretic utilities constitute the team objective. The key contributions of this thesis lie in the quantification and study of cooperative control between sensors and platforms using information as a common utility measure. In particular, * The problem of information gathering is formulated as an optimal control problem by identifying formal measures of information with utility or pay-off. * An information-theoretic utility model of coupling and coordination between decentralised decision makers is elucidated. This is used to describe how the information gathering strategies of a team of autonomous systems are coupled. * Static and dynamic information structures for team members are defined. It is shown that the use of static information structures can lead to efficient, although sub-optimal, decentralised control strategies for the team. * Significant examples in decentralised control of a team of sensors are developed. These include the multi-vehicle multi-target bearings-only tracking problem, and the area coverage or exploration problem for multiple vehicles. These examples demonstrate the range of non-trivial problems to which the theory in this thesis can be employed