Coordination Strategies for Human Supervisory Control of Robotic Teams

Autonomous mobile sensor teams are crucial to many civilian and military applications. These robotic teams often operate within a larger supervisory system, involving human operators who oversee the mission and analyze sensory data. Here, both the human and the robotic system sub-components, as well as interactions between them, must be carefully considered in designing effective mission coordination strategies. This dissertation explores a series of representative sub-problems relating to the analysis and coordination of both mobile sensors and human operators within supervisory systems. The content herein is presented in three parts: Part I focuses on coordinating operator behavior independently (operator-focused methods), Part II focuses on coordinating mobile-sensor behavior independently (sensor-focused methods), and Part III focuses on jointly coordinating both operator and mobile sensor behavior (joint methods). The content herein is primarily motivated by a particular application in which Unmanned Aerial Vehicles collect visual imagery to be analyzed by a remotely located operator, although many of the results apply to any system of similar architecture. Specifically, with regard to operator-focused methods, Chapter 2 illustrates how physiological sensing, namely eye tracking, may provide aid in modeling operator behavior and assessing the usability of user interfaces. The results of a pilot usability study in which human observers interact with a supervisory control interface are presented, and eye-tracking data is correlated with various usability metrics. Chapter 3 develops robust scheduling algorithms for determining the ordering in which operators should process sensory tasks to both boost performance and decrease variance. A scenario-based, Mixed-Integer Linear Program (MILP) framework is presented, and is assessed in a series of numerical studies. With regard to sensor-focused methods, Chapters 4 and 5 consider two types of supervisory surveillance missions:Chapter 4 develops a cloud-based coverage strategy for persistent surveillance of planar regions. The scheme operates in a dynamic environment, only requiring sporadic, unplanned data exchanges between a central cloud and the sensors in the field. The framework is shown to provide collision avoidance and, in certain cases, produce convergence to a Pareto-optimal coverage configuration. In chapter 5, a heuristic routing scheme is discussed to produce Dubins tours for persistent surveillance of discrete targets, each with associated visibility and dwell-time constraints. Under some assumptions, the problem is posed as a constrained optimization that seeks a minimum-length tour, while simultaneously constraining the time required to reach the first target. A sampling-based scheme is used to approximate solutions to the constrained optimization. This approach is also shown to have desirable resolution completeness properties.Finally, Chapter 6 explores joint methods for coordinating both operator and sensor behavior in the context of a discrete surveillance mission (similar to that of Chapter 5), in which UAVs collect imagery of static targets to be analyzed by the human operator.In particular, a method is proposed to simultaneously construct UAV routes and operator schedules, with the goal of maintaining the operator's task load within a high-performance regime and preventing unnecessary UAV loitering. The full routing/scheduling problem is posed as a mixed-integer (non-linear) program, which can be equivalently represented as a MILP through the addition of auxiliary variables. For scalability, a MILP-based receding-horizon method is proposed to incrementally construct suboptimal solutions to the full optimization problem, which can be extended using a scenario-based approach (similar to that of Chapter 3) to incorporate robustness to operator uncertainty

Multi-Robot Persistent Coverage in Complex Environments

Los recientes avances en robótica móvil y un creciente desarrollo de robots móviles asequibles han impulsado numerosas investigaciones en sistemas multi-robot. La complejidad de estos sistemas reside en el diseño de estrategias de comunicación, coordinación y controlpara llevar a cabo tareas complejas que un único robot no puede realizar. Una tarea particularmente interesante es la cobertura persistente, que pretende mantener cubierto en el tiempo un entorno con un equipo de robots moviles. Este problema tiene muchas aplicaciones como aspiración o limpieza de lugares en los que la suciedad se acumula constantemente, corte de césped o monitorización ambiental. Además, la aparición de vehículos aéreos no tripulados amplía estas aplicaciones con otras como la vigilancia o el rescate.Esta tesis se centra en el problema de cubrir persistentemente entornos progresivamente mas complejos. En primer lugar, proponemos una solución óptima para un entorno convexo con un sistema centralizado, utilizando programación dinámica en un horizonte temporalnito. Posteriormente nos centramos en soluciones distribuidas, que son más robustas, escalables y eficientes. Para solventar la falta de información global, presentamos un algoritmo de estimación distribuido con comunicaciones reducidas. Éste permite a los robots teneruna estimación precisa de la cobertura incluso cuando no intercambian información con todos los miembros del equipo. Usando esta estimación, proponemos dos soluciones diferentes basadas en objetivos de cobertura, que son los puntos del entorno en los que más se puedemejorar dicha cobertura. El primer método es un controlador del movimiento que combina un término de gradiente con un término que dirige a los robots hacia sus objetivos. Este método funciona bien en entornos convexos. Para entornos con algunos obstáculos, el segundométodo planifica trayectorias abiertas hasta los objetivos, que son óptimas en términos de cobertura. Finalmente, para entornos complejos no convexos, presentamos un algoritmo capaz de encontrar particiones equitativas para los robots. En dichas regiones, cada robotplanifica trayectorias de longitud finita a través de un grafo de caminos de tipo barrido.La parte final de la tesis se centra en entornos discretos, en los que únicamente un conjunto finito de puntos debe que ser cubierto. Proponemos una estrategia que reduce la complejidad del problema separándolo en tres subproblemas: planificación de trayectoriascerradas, cálculo de tiempos y acciones de cobertura y generación de un plan de equipo sin colisiones. Estos subproblemas más pequeños se resuelven de manera óptima. Esta solución se utiliza en último lugar para una novedosa aplicación como es el calentamiento por inducción doméstico con inductores móviles. En concreto, la adaptamos a las particularidades de una cocina de inducción y mostramos su buen funcionamiento en un prototipo real.Recent advances in mobile robotics and an increasing development of aordable autonomous mobile robots have motivated an extensive research in multi-robot systems. The complexity of these systems resides in the design of communication, coordination and control strategies to perform complex tasks that a single robot can not. A particularly interesting task is that of persistent coverage, that aims to maintain covered over time a given environment with a team of robotic agents. This problem is of interest in many applications such as vacuuming, cleaning a place where dust is continuously settling, lawn mowing or environmental monitoring. More recently, the apparition of useful unmanned aerial vehicles (UAVs) has encouraged the application of the coverage problem to surveillance and monitoring. This thesis focuses on the problem of persistently covering a continuous environment in increasingly more dicult settings. At rst, we propose a receding-horizon optimal solution for a centralized system in a convex environment using dynamic programming. Then we look for distributed solutions, which are more robust, scalable and ecient. To deal with the lack of global information, we present a communication-eective distributed estimation algorithm that allows the robots to have an accurate estimate of the coverage of the environment even when they can not exchange information with all the members of the team. Using this estimation, we propose two dierent solutions based on coverage goals, which are the points of the environment in which the coverage can be improved the most. The rst method is a motion controller, that combines a gradient term with a term that drives the robots to the goals, and which performs well in convex environments. For environments with some obstacles, the second method plans open paths to the goals that are optimal in terms of coverage. Finally, for complex, non-convex environments we propose a distributed algorithm to nd equitable partitions for the robots, i.e., with an amount of work proportional to their capabilities. To cover this region, each robot plans optimal, nite-horizon paths through a graph of sweep-like paths. The nal part of the thesis is devoted to discrete environment, in which only a nite set of points has to be covered. We propose a divide-and-conquer strategy to separate the problem to reduce its complexity into three smaller subproblem, which can be optimally solved. We rst plan closed paths through the points, then calculate the optimal coverage times and actions to periodically satisfy the coverage required by the points, and nally join together the individual plans of the robots into a collision-free team plan that minimizes simultaneous motions. This solution is eventually used for a novel application that is domestic induction heating with mobile inductors. We adapt it to the particular setting of a domestic hob and demonstrate that it performs really well in a real prototype.<br /

Optimal Routing of Unmanned Vehicles in Persistent Monitoring Missions

Missions such as forest fire monitoring, military surveillance and infrastructure monitoring are referred to as persistent monitoring missions. These missions rely heavily on continual data collection from various locations, referred to as targets. In this dissertation, we consider a framework in which data is collected from the targets with the aid of unmanned aerial vehicles (UAVs). A UAV makes a physical visit to the targets for data collection, and immediately transmits the collected data to a base station for further analysis. Typically, the duration of these monitoring missions is long, and the monitoring vehicles are required to stay in flight for extended periods of time. Therefore, the batteries powering the UAVs must be recharged regularly at a recharging station/depot. From utilitarian and economic points of view, an efficient execution of these missions calls for two requisites: 1) minimizing the time delay between successive data collections at targets; 2) maximizing the total charge/energy drawn from batteries. The maximum time delay between successive data collections at any target is characterized by a function referred to as the walk revisit time, or simply the revisit time. Given a set of targets and a UAV tasked with monitoring the targets, the charge capacity of the battery powering the UAV can be surrogated by the number of visits the UAV can make to the targets without requiring a recharge. To minimize the wastage of energy resources, a charge penalty is imposed on the visits that are unutilized before each recharge. The aim of this work is to find optimal routes for the UAV(s) to visit the targets such that the sum of the revisit time and the charge penalty is minimized. The optimal route planning problem is determined by a number of factors such as the number of UAVs used for monitoring, the aerial platform on which the monitoring UAVs are built, the location of their depots, relative importance of the targets being monitored, etc. In this dissertation, we focus on equally weighted targets and address four different variants of the problem, all of which are computationally extremely challenging. The variants considered are the following: 1) single UAV with no motion constraints and the depot located at one of the targets; 2) single UAV with curvature constraints on its path and the depot located at one of the targets; 3) single UAV with no motion constraints and its depot stationed at a location different from that of the targets; 4) multiple UAVs with no motion constraints with their depots located at the targets. This dissertation builds on the results of Variant 1; specifically, the characterization of the optimal solutions proved in this dissertation is the main contribution of this dissertation; it lends itself to a new formulation of the same problem that results in significant computational savings. The structural characterization also holds for Variant 2. Inspired by this result, conjectures are provided for the structure of optimal solution for variant 3 and is backed up by extensive numerical simulations. Variant 3 can also be perceived as a special case of targets with different weights/priorities, and therefore, the results developed in this dissertation can potentially be extended to solve a few special cases of the general problem involving arbitrarily weighted targets

Multi-robot deployment planning in communication-constrained environments

A lo largo de los últimos años se ha podido observar el aumento del uso de equipos de robots en tareas en las cuales es imposible o poco eficiente la intervención de los humanos, e incluso que implica un cierto grado de riesgo para una persona. Por ejemplo, monitorización de entornos de difícil acceso, como podrían ser túneles, minas, etc. Éste es el tema en el que se ha enfocado el trabajo realizado durante esta tesis: la planificación del despliegue de un equipo de agentes para la monitorización de entornos.La misión de los agentes es alcanzar unas localizaciones de interés y transmitirle la información observada a una estación base estática. Ante la ausencia de una infraestructura de comunicaciones, una transmisión directa a la base es imposible. Por tanto, los agentes se deben coordinar de manera autónoma, de modo que algunos de ellos alcancen los objetivos y otros realicen la función de repetidor para retransmitir la información.Nos hemos centrado en dos líneas de investigación principales, relacionadas con dos maneras del envío de la información a la estación base. En el primer enfoque, los agentes deben mantener un enlace de comunicación con la base en el momento de alcanzar los objetivos. Con el fin de, por ejemplo, poder interactuar desde la base con un robot que ha alcanzado el objetivo. Para ello hemos desarrollado un método que obtiene las posiciones óptimas para los agentes utilizados a modo de repetidor. A continuación, hemos implementado un método de planificación de caminos de modo que los agentes pudiesen navegar el máximo tiempo posible dentro de zonas con señal. Empleando conjuntamente ambos métodos, los agentes extienden el área de cobertura de la estación base, estableciendo un enlace de comunicación desde la misma hasta los objetivos marcados.Utilizando este método, el equipo es capaz de lidiar con variaciones del entorno si la comunicación entre los agentes no se pierde. Sin embargo, los eventos tan comunes e irrelevantes para los seres humanos, como el simple cierre de una puerta, pueden llegar a ser críticos para el equipo de robots. Ya que esto podría interrumpir la comunicación entre el equipo. Por ello, hemos propuesto un método distribuido para que el equipo sea capaz de reconectarse, formando una cadena hacia un objetivo, en escenarios donde haya variaciones con respecto al mapa inicial que poseían los robots.La segunda parte de la presente tesis se ha centrado en misiones de recopilación de datos de un entorno. Aquí la comunicación con la estación base, en el instante de alcanzar un objetivo, no es necesaria y a menudo imposible. Por tanto, en este tipo de escenarios, es más eficiente que algunos agentes, llamados trabajadores, recopilen datos del entorno, y otros, denominados colectores, reúnan la información de los que trabajan para periódicamente retransmitirla a la base. De este modo tan solo los colectores realizan largos viajes a la estación base, mientras que los trabajadores emplean la mayor parte de su tiempo exclusivamente a la recopilación de datos.Primero, hemos desarrollado dos métodos para la planificación de caminos para la sincronización entre los trabajadores y colectores. El primero, muestrea el espacio de manera aleatoria, para obtener una solución lo más rápido posible. El segundo, usando FMM, es más lento, pero obtiene soluciones óptimas.Finalmente, hemos propuesto una técnica global para la misión de recopilación de datos. Este método consiste en: encontrar el mejor balance entre la cantidad de trabajadores y colectores, la mejor división del escenario en áreas de trabajo para los trabajadores, la asociación de los trabajadores para transmitir los datos recopilados a los colectores o directamente a la estación base, así como los caminos de los colectores. El método propuesto trata de encontrar la mejor solución con el fin de entregar la mayor cantidad de datos y que el tiempo de "refresco" de los mismos sea el menor posible.<br /

Full Proceedings, 2018

Full conference proceedings for the 2018 International Building Physics Association Conference hosted at Syracuse University