A Planning Pipeline for Large Multi-Agent Missions

In complex multi-agent applications, human operators are often tasked with planning and managing large heterogeneous teams of humans and autonomous vehicles. Although the use of these autonomous vehicles broadens the scope of meaningful applications, many of their systems remain unintuitive and difficult to master for human operators whose expertise lies in the application domain and not at the platform level. Current research focuses on the development of individual capabilities necessary to plan multi-agent missions of this scope, placing little emphasis on the integration of these components in to a full pipeline. The work presented in this paper presents a complete and user-agnostic planning pipeline for large multiagent missions known as the HOLII GRAILLE. The system takes a holistic approach to mission planning by integrating capabilities in human machine interaction, flight path generation, and validation and verification. Components modules of the pipeline are explored on an individual level, as well as their integration into a whole system. Lastly, implications for future mission planning are discussed

Multi-Operator Gesture Control of Robotic Swarms Using Wearable Devices

The theory and design of effective interfaces for human interaction with multi-robot systems has recently gained significant interest. Robotic swarms are multi-robot systems where local interactions between robots and neighbors within their spatial neighborhood generate emergent collective behaviors. Most prior work has studied interfaces for human interaction with remote swarms, but swarms also have great potential in applications working alongside humans, motivating the need for interfaces for local interaction. Given the collective nature of swarms, human interaction may occur at many levels of abstraction ranging from swarm behavior selection to teleoperation. Wearable gesture control is an intuitive interaction modality that can meet this requirement while keeping operator hands usually unencumbered. In this paper, we present an interaction method using a gesture-based wearable device with a limited number of gestures for robust control of a complex system: a robotic swarm. Experiments conducted with a real robot swarm compare performance in single and two-operator conditions illustrating the effectiveness of the method. Results show human operators using our interaction method are able to successfully complete the task in all trials, illustrating the effectiveness of the method, with better performance in the two-operator condition, indicating separation of function is beneficial for our method. The primary contribution of our work is the development and demonstration of interaction methods that allow robust control of a difficult to understand multi robot system using only the noisy inputs typical of smartphones and other on-body sensor driven devices

Human-in-the-loop Planning and Monitoring of Swarm Search and Service Missions

As vehicles move in and out of the swarm, the amount of area seen by the swarm at any given time – coverage rate – changes dynamically. The dynamically changing coverage rate causes the arrival rate of jobs to also change dynamically. Since jobs appear only when they are sensed, predicting how and when the arrival rates change is challenging, making it difficult for operators to plan and manage SSS missions. This paper presents a user study that explores the efficacy and ease-of-use of a prediction model – Hybrid Model – as an aid for operators tasked with planning and monitoring SSS missions where the arrival rate of jobs changes dynamicall

Hybrid Model for A Priori Performance Prediction of Multi-Job Type Swarm Search and Service Missions

In Swarm Search and Service (SSS) applications, swarm vehicles are responsible for concurrently searching an area while immediately servicing jobs discovered while searching. Multiple job types may be present in the environment. As vehicles move in and out of the swarm to service jobs, the coverage rate (i.e., area searched by the swarm per time step) changes dynamically to reflect the number of vehicles currently engaged in search. As a result, the arrival rates of jobs also changes dynamically. When planning SSS missions, the resource requirements, such as the swarm size needed to achieve a desired system performance, must be determined. The dynamically changing arrival rates make traditional queuing methods ill-suited to predict the performance of the swarm. This paper presents a hybrid method - Hybrid Model - for predicting the performance of the swarm a priori. It utilizes a Markov model, whose state representation captures the proportion of agents searching or servicing jobs. State-dependent queuing models are used to calculate the state transition function of the Markov states. The model has been developed as a prediction tool to assist mission planners in balancing complex trade-offs, but also provides a basis for optimizing swarm size where cost functions are known. The Hybrid Model is tested in previously considered constant coverage rate scenarios and the results are compared to a previously developed Queuing Model. Additional SSS missions are then simulated and their resulting performance is used to further evaluate the effectiveness of using the Hybrid Model as a prediction tool for swarm performance in more general scenarios with dynamically changing coverage rates

Determining Effective Swarm Sizes for Multi-Job Type Missions

Swarm search and service (SSS) missions require large swarms to simultaneously search an area while servicing jobs as they are encountered. Jobs must be immediately serviced and can be one of several different job types - each requiring a different service time and number of vehicles to complete its service successfully. After jobs are serviced, vehicles are returned to the swarm and become available for reallocation. As part of SSS mission planning, human operators must determine the number of vehicles needed to achieve this balance. The complexities associated with balancing vehicle allocation to multiple as yet unknown tasks with returning vehicles makes this extremely difficult for humans. Previous work assumes that all system jobs are known ahead of time or that vehicles move independently of each other in a multi-agent framework. We present a dynamic vehicle routing (DVR) framework whose policies optimally allocate vehicles as jobs arrive. By incorporating time constraints into the DVR framework, an M/M/k/k queuing model can be used to evaluate overall steady state system performance for a given swarm size. Using these estimates, operators can rapidly compare system performance across different configurations, leading to more effective choices for swarm size. A sensitivity analysis is performed and its results are compared with the model, illustrating the appropriateness of our method to problems of plausible scale and complexity

Planning and Monitoring Multi-Job Type Swarm Search and Service Missions

To transition from control theory to real applications, it is important to study missions such as Swarm Search and Service (SSS) where vehicles are not only required to search an area, but also service all jobs that they find. In SSS missions, each type of job requires a group of vehicles to break off from the swarm for a given amount of time to service it. The required number of vehicles and the service rate are unique to each job type. Once a job has been completed, the vehicles are able to return to the swarm for use elsewhere. If not enough vehicles are present in the swarm at the time that the job is identified, that job is dropped without being serviced. In SSS missions that occur in open environments, the arrival rate of jobs varies dynamically as vehicles move in and out of the swarm to service jobs. Human operators are tasked with effectively planning and managing these complex missions. This paper presents a user study that seeks to test the efficacy and ease-of-use of a prediction model known as the Hybrid Model as an aid in planning and monitoring tasks. Results show that the novel computational model aid allows operators to more effectively choose the necessary swarm size to handle expected mission workload, as well as, maintain sufficient situation awareness to evaluate the performance of the swarm during missions

Decentralized Method for Sub-swarm Deployment and Rejoining

As part of swarm search and service (SSS) missions, robots are tasked with servicing jobs as they are sensed. This requires small sub-swarm teams to leave the swarm for a specified amount of time to service the jobs. In doing so, fewer robots are required to change motion than if the whole swarm were diverted, thereby minimizing the job’s overall effect on the swarm’s main goal. We explore the problem of removing the required number of robots from the swarm, while maintaining overall swarm connectivity. By preserving connectivity, robots are able to successfully rejoin the swarm upon completion of their assigned job. These robots are then made available for reallocation. We propose a decentralized and asynchronous method for breaking off sub-swarm groups and rejoining them with the main swarm using the swarm’s communication graph topology. Both single and multiple job site cases are explored. The results are compared against a full swarm movement method. Simulation results show that the proposed method outperforms a full swarm method in the average number of messages sent per robot in each step, as well as, the distance traveled by the swarm

A Safe Cooperative Framework for Atmospheric Science Missions with Multiple Heterogeneous UAS using Piecewise Bezier Curves

Autonomous operation of UAS holds promise for greater productivity of atmospheric science missions. However, several challenges need to be overcome before such missions can be made autonomous. This paper presents a framework for safe autonomous operations of multiple vehicles, particularly suited for atmospheric science missions. The framework revolves around the use of piecewise Bezier curves for trajectory representation, which in conjunction with path-following and time-coordination algorithms, allows for safe coordinated operations of multiple vehicles

Human-in-the-loop Mission Planning and Monitoring for Robot Swarms

Robot swarms are large multi-robot systems that use simple, local control laws to produce global emergent behaviors. They are able to self-organizeand coordinate without the use of a centralized mechanism to accomplish tasks otherwise unachievable by a single individual (e.g., in-situ correlative atmospheric data collection). Due to their use of information obtained only from their direct neighbors, these systems are robust to individual robot failures and insertions or removals of swarm members. As a result, robot swarms are scalable.Their inherent scalability and robustness makes robot swarms suitable for many applications such as search and rescue and surveillance. The work in this thesis focuses on applications known as Swarm Search and Service (SSS) Missions. In SSS missions, which naturally arise from foraging tasks such as search and rescue, the swarm is required to simultaneously search an area while servicing jobs as they are encountered. Jobs must be immediately serviced and can be one of several different job types– each requiring a different service time and number of vehicles to complete its service successfully. After jobs are serviced, vehicles are returned to the swarm and become available for reallocation. As part of SSS mission planning, human operators must determine the number of vehiclesneeded to achieve this balance. The complexities associated with balancing vehicle allocation to multiple as yet unknown tasks with returning vehicles makes this extremely difficult for humans. Previous work assumesthat all system jobs are known ahead of time or that vehicles move independently of each other in a multi-agent framework. This thesis explores the topic of human-in-the-loop mission planning and monitoring for SSS missions. Natural language-based interfaces are designed for intuitive mission definition. Two models are developed to predict the performance of the swarm: the Queuing Model and the Hybrid Model. The Queuing Model is able to predict the performance of the swarm for missions where the swarm movement is constrained (e.g., urban) and the coverage rate of the swarm remains constant while the Hybrid Model builds upon principles in the Queuing Model to handle additional open environments scenarios where the coverage rate dynamically changes with the size of the swarm. These models, when given to human operators, act as a planning tool aid. Operators can rapidly compare system performance across different configurations, leading to more effective mission plans and improved performance. In addition, the Hybrid Model is able to aid operators in maintaining an accurate, real-time situational awarenessof the mission, thereby allowing operators to determine how well the mission is going and if/what errors are occurring. Lastly, to effectively carry out SSS missions, this thesis presents a decentralized method for breaking off robots to reach multiple job sites and rejoining them with the swarmonce service is completed. <br