Path planning with spatiotemporal optimal stopping for stochastic mission monitoring

© 2017 IEEE. We consider an optimal stopping formulation of the mission monitoring problem, in which a monitor vehicle must remain in close proximity to an autonomous robot that stochastically follows a predicted trajectory. This problem arises in a diverse range of scenarios, such as autonomous underwater vehicles supervised by surface vessels, pedestrians monitored by aerial vehicles, and animals monitored by agricultural robots. The key problem characteristics we consider are that the monitor must remain stationary while observing the robot, robot motion is modeled in general as a stochastic process, and observations are modeled as a spatial probability distribution. We propose a resolution-complete algorithm that runs in a polynomial time. The algorithm is based on a sweep-plane approach and generates a motion plan that maximizes the expected observation time and value. A variety of stochastic models may be used to represent the robot trajectory. We present results with data drawn from real AUV missions, a real pedestrian trajectory dataset and Monte Carlo simulations. Our results demonstrate the performance and behavior of our algorithm, and relevance to a variety of applications

The design and implementation of a multi-agent architecture to increase coordination efficiency in multi-AUV operations

This research addresses the problem of coordinating multiple autonomous underwater vehicle (AUV) operations. An intelligent mission executive has been created that uses multi-agent technology to control and coordinate multiple AUVs in communication deficient environments. By incorporating real time vehicle prediction, blackboardbased hierarchical mission plans and mission optimisation in conjunction with a simple broadcast communication system this system aims to handle the limitations inherent in underwater operations and intelligently control multiple vehicles. In this research efficiency is evaluated and then compared to the current state of the art in multiple AUV control. The research is then validated in real AUV coordination trials. Results will show that compared to the state of the art the control system developed and implemented in this research coordinates multiple vehicles more efficiently and is able to function in a range of poor communication environments. These findings are supported by in water validation trials with heterogeneous AUVs. This thesis will first present an in depth state of the art of the related research topics including multi-agent systems, collaborative robotics and autonomous underwater vehicles. The development and functionality of this research will then be explained followed by a detailed description of the experiments. Results are then presented both for the simulated and real world trials followed by a discussion of the findings

Algorithms for multi-robot systems on the cooperative exploration & last-mile delivery problems

La aparición de los vehículos aéreos no tripulados (UAVs) y de los vehículos terrestres no tripulados (UGVs) ha llevado a la comunidad científica a enfrentarse a problemas ideando paradigmas de cooperación con UGVs y UAVs. Sin embargo, no suele ser trivial determinar si la cooperación entre UGVs y UAVs es adecuada para un determinado problema. Por esta razón, en esta tesis, investigamos un paradigma particular de cooperación UGV-UAV en dos problemas de la literatura, y proponemos un controlador autónomo para probarlo en escenarios simulados. Primero, formulamos un problema particular de exploración cooperativa que consiste en alcanzar un conjunto de puntos de destino en un área de exploración a gran escala. Este problema define al UGV como una estación de carga móvil para transportar el UAV a través de diferentes lugares desde donde el UAV puede alcanzar los puntos de destino. Por consiguiente, proponemos el algoritmo TERRA para resolverlo. Este algoritmo se destaca por dividir el problema de exploración en cinco subproblemas, en los que cada subproblema se resuelve en una etapa particular del algoritmo. Debido a la explosión de la entrega de paquetes en las empresas de comercio electrónico, formulamos también una generalización del conocido problema de la entrega en la última milla. En este caso, el UGV actúa como una estación de carga móvil que transporta a los paquetes y a los UAVs, y estos se encargan de entregarlos. De esta manera, seguimos la estrategia de división descrita por TERRA, y proponemos el algoritmo COURIER. Este algoritmo replica las cuatro primeras etapas de TERRA, pero construye una nueva quinta etapa para producir un plan de tareas que resuelva el problema. Para evaluar el paradigma de cooperación UGV-UAV en escenarios simulados, proponemos el controlador autónomo ARIES. Este controlador sigue un enfoque jerárquico descentralizado de líder-seguidor para integrar cualquier paradigma de cooperación de manera distribuida. Ambos algoritmos han sido caracterizados para identificar los aspectos relevantes del paradigma de cooperación en los problemas relacionados. Además, ambos demuestran un gran rendimiento del paradigma de cooperación en tales problemas, y al igual que el controlador autónomo, revelan un gran potencial para futuras aplicaciones reales.The emergence of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) has conducted the research community to face historical complex problems by devising UGV-UAV cooperation paradigms. However, it is usually not a trivial task to determine whether or not a UGV-UAV cooperation is suitable for a particular problem. For this reason, in this thesis, we investigate a particular UGV-UAV cooperation paradigm over two problems in the literature, and we propose an autonomous controller to test it on simulated scenarios. Driven by the planetary exploration, we formulate a particular cooperative exploration problem consisting of reaching a set of target points in a large-scale exploration area. This problem defines the UGV as a moving charging station to carry the UAV through different locations from where the UAV can reach the target points. Consequently, we propose the cooperaTive ExploRation Routing Algorithm (TERRA) to solve it. This algorithm stands out for splitting up the exploration problem into five sub-problems, in which each sub-problem is solved in a particular stage of the algorithm. In the same way, driven by the explosion of parcels delivery in e-commerce companies, we formulate a generalization of the well-known last-mile delivery problem. This generalization defines the same UGV’s and UAV’s rol as the exploration problem. That is, the UGV acts as a moving charging station which carries the parcels along several UAVs to deliver them. In this way, we follow the split strategy depicted by TERRA to propose the COoperative Unmanned deliveRIEs planning algoRithm (COURIER). This algorithm replicates the first four TERRA’s stages, but it builds a new fifth stage to produce a task plan solving the problem. In order to evaluate the UGV-UAV cooperation paradigm on simulated scenarios, we propose the Autonomous coopeRatIve Execution System (ARIES). This controller follows a hierarchical decentralized leader-follower approach to integrate any cooperation paradigm in a distributed manner. Both algorithms have been characterized to identify the relevant aspects of the cooperation paradigm in the related problems. Also, both of them demonstrate a great performance of the cooperation paradigm in such problems, and as well as the autonomous controller, reveal a great potential for future real applications

Reitinsuunnittelu määrätyssä järjestyksessä tehtäville peltotöille usean työkoneen yhteistyönä

Coverage path planning is the task of finding a collision free path that passes over every point of an area or volume of interest. In agriculture, the coverage task is encountered especially in the process of crop cultivation. Several tasks are performed on the field, one after the other, during the cultivation cycle. Cooperation means that multiple agents, in this case vehicles, are working together towards a common goal. Several studies consider the problem where a single task is divided and assigned among the agents. In this thesis, however, the vehicles have different tasks that are sequentially dependent, that is, the first task must be completed before the other. The tasks are performed simultaneously on the same area. The literature review suggests that there is a lack of previous research on this topic. The objective of this thesis was to develop an algorithm to solve the cooperative coverage path planning problem for sequentially dependent tasks. A tool chain that involves Matlab, Simulink and Visual Studio was adapted for the development and testing of the solution. A development and testing architecture was designed including a compatible interface to a simulation and a real-life test environment. Two different algorithms were implemented based on the idea of computing short simultaneous paths at a time and scheduling them in real-time. The results were successfully demonstrated in a real-life test environment with two tractors equipped with a disc cultivator and a seeder. The objective was to sow the test area. The test drives show that with the algorithms that were developed in this thesis it is possible to perform two sequentially dependent agricultural coverage tasks simultaneously on the same area.Kattavassa reitinsuunnittelussa yritetään löytää polku, jonka aikana määritelty ala tai tilavuus tulee käytyä läpi niin että alueen jokainen piste on käsitelty. Maataloudessa tämä tehtävä on merkityksellinen erityisesti peltoviljelyssä. Useita peltotöitä suoritetaan yksi toisensa jälkeen samalla alueella viljelyvuoden aikana. Useissa tutkimuksissa käsitellään yhteistyönä tehtävää reitinsuunnittelua, jossa yksi tehtävä on jaettu osiin ja osat jaetaan useiden tekijöiden kuten robottien kesken. Tässä diplomityössä peltotyökoneilla on kuitenkin omat erilliset tehtävänsä, joilla on määrätty järjestys, eli niiden suorittaminen riippuu työjärjestyksestä. Työkoneet työskentelevät samanaikaisesti samalla alueella. Diplomityössä tehty kirjallisuuskatsaus viittaa siihen, että vastaavaa aihetta ei ole aiemmin tutkittu. Tämän diplomityön tavoitteena on kehittää algoritmi, jolla voidaan toteuttaa reitinsuunnittelu määrätyssä järjestyksessä tehtäville peltotöille usean peltotyökoneen yhteistyönä. Algoritmikehitystä ja testausta varten suunniteltiin yhtenäinen rajapinta, jolla algoritmia voitaisiin testata sekä simulaatiossa että todellisessa testitilanteessa. Algoritmikehityksessä käytettiin työkaluina Matlab, Simulink ja Visual Studio -ohjelmia. Työssä toteutettiin kaksi algoritmia, jotka perustuvat samaan ideaan: suunnitellaan kerrallaan kaksi lyhyttä samanaikaista polkua, jotka ajoitetaan reaaliajassa. Algoritmeja testattiin todellisessa testiympäristössä kahden työkoneen yhteistyönä, kun tavoitteena on kylvää koko testialue. Ensimmäinen työvaihe suoritettiin lautasmuokkaimella ja toinen kylvökoneella. Testiajot osoittavat, että diplomityössä kehitetyillä algoritmeilla voidaan ohjata kahden toisistaan riippuvaisen peltotyön toteutus samanaikaisesti samalla peltoalueella

A Multiagent Deep Reinforcement Learning Approach for Path Planning in Autonomous Surface Vehicles: The Ypacaraí Lake Patrolling Case

Article number 9330612Autonomous surfaces vehicles (ASVs) excel at monitoring and measuring aquatic nutrients due to their autonomy, mobility, and relatively low cost. When planning paths for such vehicles, the task of patrolling with multiple agents is usually addressed with heuristics approaches, such as Reinforcement Learning (RL), because of the complexity and high dimensionality of the problem. Not only do efficient paths have to be designed, but addressing disturbances in movement or the battery’s performance is mandatory. For this multiagent patrolling task, the proposed approach is based on a centralized Convolutional Deep Q-Network, designed with a final independent dense layer for every agent to deal with scalability, with the hypothesis/assumption that every agent has the same properties and capabilities. For this purpose, a tailored reward function is created which penalizes illegal actions (such as collisions) and rewards visiting idle cells (cells that remains unvisited for a long time). A comparison with various multiagent Reinforcement Learning (MARL) algorithms has been done (Independent Q-Learning, Dueling Q-Network and multiagent Double Deep Q-Learning) in a case-study scenario like the Ypacaraí lake in Asunción (Paraguay). The training results in multiagent policy leads to an average improvement of 15% compared to lawn mower trajectories and a 6% improvement over the IDQL for the case-study considered. When evaluating the training speed, the proposed approach runs three times faster than the independent algorithm.Ministerio de Ciencia, Innovación y Universidades (España) RTI2018-098964-B-I00Junta de Andalucía(España) PY18-RE000