A Visual Analytics Approach to Debugging Cooperative, Autonomous Multi-Robot Systems' Worldviews

Autonomous multi-robot systems, where a team of robots shares information to perform tasks that are beyond an individual robot's abilities, hold great promise for a number of applications, such as planetary exploration missions. Each robot in a multi-robot system that uses the shared-world coordination paradigm autonomously schedules which robot should perform a given task, and when, using its worldview--the robot's internal representation of its belief about both its own state, and other robots' states. A key problem for operators is that robots' worldviews can fall out of sync (often due to weak communication links), leading to desynchronization of the robots' scheduling decisions and inconsistent emergent behavior (e.g., tasks not performed, or performed by multiple robots). Operators face the time-consuming and difficult task of making sense of the robots' scheduling decisions, detecting de-synchronizations, and pinpointing the cause by comparing every robot's worldview. To address these challenges, we introduce MOSAIC Viewer, a visual analytics system that helps operators (i) make sense of the robots' schedules and (ii) detect and conduct a root cause analysis of the robots' desynchronized worldviews. Over a year-long partnership with roboticists at the NASA Jet Propulsion Laboratory, we conduct a formative study to identify the necessary system design requirements and a qualitative evaluation with 12 roboticists. We find that MOSAIC Viewer is faster- and easier-to-use than the users' current approaches, and it allows them to stitch low-level details to formulate a high-level understanding of the robots' schedules and detect and pinpoint the cause of the desynchronized worldviews.Comment: To appear in IEEE Conference on Visual Analytics Science and Technology (VAST) 202

Command and Control Systems for Search and Rescue Robots

The novel application of unmanned systems in the domain of humanitarian Search and Rescue (SAR) operations has created a need to develop specific multi-Robot Command and Control (RC2) systems. This societal application of robotics requires human-robot interfaces for controlling a large fleet of heterogeneous robots deployed in multiple domains of operation (ground, aerial and marine). This chapter provides an overview of the Command, Control and Intelligence (C2I) system developed within the scope of Integrated Components for Assisted Rescue and Unmanned Search operations (ICARUS). The life cycle of the system begins with a description of use cases and the deployment scenarios in collaboration with SAR teams as end-users. This is followed by an illustration of the system design and architecture, core technologies used in implementing the C2I, iterative integration phases with field deployments for evaluating and improving the system. The main subcomponents consist of a central Mission Planning and Coordination System (MPCS), field Robot Command and Control (RC2) subsystems with a portable force-feedback exoskeleton interface for robot arm tele-manipulation and field mobile devices. The distribution of these C2I subsystems with their communication links for unmanned SAR operations is described in detail. Field demonstrations of the C2I system with SAR personnel assisted by unmanned systems provide an outlook for implementing such systems into mainstream SAR operations in the future

Evolutionary Robot Swarms Under Real-World Constraints

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, na especialidade de Automação e Robótica, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Faculdade de Ciências e Tecnologia da Universidade de CoimbraNas últimas décadas, vários cientistas e engenheiros têm vindo a estudar as estratégias provenientes da natureza. Dentro das arquiteturas biológicas, as sociedades que vivem em enxames revelam que agentes simplistas, tais como formigas ou pássaros, são capazes de realizar tarefas complexas usufruindo de mecanismos de cooperação. Estes sistemas abrangem todas as condições necessárias para a sobrevivência, incorporando comportamentos de cooperação, competição e adaptação. Na “batalha” sem fim em prol do progresso dos mecanismos artificiais desenvolvidos pelo homem, a ciência conseguiu simular o primeiro comportamento em enxame no final dos anos oitenta. Desde então, muitas outras áreas, entre as quais a robótica, beneficiaram de mecanismos de tolerância a falhas inerentes da inteligência coletiva de enxames. A área de investigação deste estudo incide na robótica de enxame, consistindo num domínio particular dos sistemas robóticos cooperativos que incorpora os mecanismos de inteligência coletiva de enxames na robótica. Mais especificamente, propõe-se uma solução completa de robótica de enxames a ser aplicada em contexto real. Nesta ótica, as operações de busca e salvamento foram consideradas como o caso de estudo principal devido ao nível de complexidade associado às mesmas. Tais operações ocorrem tipicamente em cenários dinâmicos de elevadas dimensões, com condições adversas que colocam em causa a aplicabilidade dos sistemas robóticos cooperativos. Este estudo centra-se nestes problemas, procurando novos desafios que não podem ser ultrapassados através da simples adaptação da literatura da especialidade em algoritmos de enxame, planeamento, controlo e técnicas de tomada de decisão. As contribuições deste trabalho sustentam-se em torno da extensão do método Particle Swarm Optimization (PSO) aplicado a sistemas robóticos cooperativos, denominado de Robotic Darwinian Particle Swarm Optimization (RDPSO). O RDPSO consiste numa arquitetura robótica de enxame distribuída que beneficia do particionamento dinâmico da população de robôs utilizando mecanismos evolucionários de exclusão social baseados na sobrevivência do mais forte de Darwin. No entanto, apesar de estar assente no caso de estudo do RDPSO, a aplicabilidade dos conceitos aqui propostos não se encontra restrita ao mesmo, visto que todos os algoritmos parametrizáveis de enxame de robôs podem beneficiar de uma abordagem idêntica. Os fundamentos em torno do RDPSO são introduzidos, focando-se na dinâmica dos robôs, nos constrangimentos introduzidos pelos obstáculos e pela comunicação, e nas suas propriedades evolucionárias. Considerando a colocação inicial dos robôs no ambiente como algo fundamental para aplicar sistemas de enxames em aplicações reais, é assim introduzida uma estratégia de colocação de robôs realista. Para tal, a população de robôs é dividida de forma hierárquica, em que são utilizadas plataformas mais robustas para colocar as plataformas de enxame no cenário de forma autónoma. Após a colocação dos robôs no cenário, é apresentada uma estratégia para permitir a criação e manutenção de uma rede de comunicação móvel ad hoc com tolerância a falhas. Esta estratégia não considera somente a distância entre robôs, mas também a qualidade do nível de sinal rádio frequência, redefinindo assim a sua aplicabilidade em cenários reais. Os aspetos anteriormente mencionados estão sujeitos a uma análise detalhada do sistema de comunicação inerente ao algoritmo, para atingir uma implementação mais escalável do RDPSO a cenários de elevada complexidade. Esta elevada complexidade inerente à dinâmica dos cenários motivaram a ultimar o desenvolvimento do RDPSO, integrando para o efeito um mecanismo adaptativo baseado em informação contextual (e.g., nível de atividade do grupo). Face a estas considerações, o presente estudo pode contribuir para expandir o estado-da-arte em robótica de enxame com algoritmos inovadores aplicados em contexto real. Neste sentido, todos os métodos propostos foram extensivamente validados e comparados com alternativas, tanto em simulação como com robôs reais. Para além disso, e dadas as limitações destes (e.g., número limitado de robôs, cenários de dimensões limitadas, constrangimentos reais limitados), este trabalho contribui ainda para um maior aprofundamento do estado-da-arte, onde se propõe um modelo macroscópico capaz de capturar a dinâmica inerente ao RDPSO e, até certo ponto, estimar analiticamente o desempenho coletivo dos robôs perante determinada tarefa. Em suma, esta investigação pode ter aplicabilidade prática ao colmatar a lacuna que se faz sentir no âmbito das estratégias de enxames de robôs em contexto real e, em particular, em cenários de busca e salvamento.Over the past decades, many scientists and engineers have been studying nature’s best and time-tested patterns and strategies. Within the existing biological architectures, swarm societies revealed that relatively unsophisticated agents with limited capabilities, such as ants or birds, were able to cooperatively accomplish complex tasks necessary for their survival. Those simplistic systems embrace all the conditions necessary to survive, thus embodying cooperative, competitive and adaptive behaviours. In the never-ending battle to advance artificial manmade mechanisms, computer scientists simulated the first swarm behaviour designed to mimic the flocking behaviour of birds in the late eighties. Ever since, many other fields, such as robotics, have benefited from the fault-tolerant mechanism inherent to swarm intelligence. The area of research presented in this Ph.D. Thesis focuses on swarm robotics, which is a particular domain of multi-robot systems (MRS) that embodies the mechanisms of swarm intelligence into robotics. More specifically, this Thesis proposes a complete swarm robotic solution that can be applied to real-world missions. Although the proposed methods do not depend on any particular application, search and rescue (SaR) operations were considered as the main case study due to their inherent level of complexity. Such operations often occur in highly dynamic and large scenarios, with harsh and faulty conditions, that pose several problems to MRS applicability. This Thesis focuses on these problems raising new challenges that cannot be handled appropriately by simple adaptation of state-of-the-art swarm algorithms, planning, control and decision-making techniques. The contributions of this Thesis revolve around an extension of the Particle Swarm Optimization (PSO) to MRS, denoted as Robotic Darwinian Particle Swarm Optimization (RDPSO). The RDPSO is a distributed swarm robotic architecture that benefits from the dynamical partitioning of the whole swarm of robots by means of an evolutionary social exclusion mechanism based on Darwin’s survival-of-the-fittest. Nevertheless, although currently applied solely to the RDPSO case study, the applicability of all concepts herein proposed is not restricted to it, since all parameterized swarm robotic algorithms may benefit from a similar approach The RDPSO is then proposed and used to devise the applicability of novel approaches. The fundamentals around the RDPSO are introduced by focusing on robots’ dynamics, obstacle avoidance, communication constraints and its evolutionary properties. Afterwards, taking the initial deployment of robots within the environment as a basis for applying swarm robotics systems into real-world applications, the development of a realistic deployment strategy is proposed. For that end, the population of robots is hierarchically divided, wherein larger support platforms autonomously deploy smaller exploring platforms in the scenario, while considering communication constraints and obstacles. After the deployment, a way of ensuring a fault-tolerant multi-hop mobile ad hoc communication network (MANET) is introduced to explicitly exchange information needed in a collaborative realworld task execution. Such strategy not only considers the maximum communication range between robots, but also the minimum signal quality, thus refining the applicability to real-world context. This is naturally followed by a deep analysis of the RDPSO communication system, describing the dynamics of the communication data packet structure shared between teammates. Such procedure is a first step to achieving a more scalable implementation by optimizing the communication procedure between robots. The highly dynamic characteristics of real-world applications motivated us to ultimate the RDPSO development with an adaptive strategy based on a set of context-based evaluation metrics. This thesis contributes to the state-of-the-art in swarm robotics with novel algorithms for realworld applications. All of the proposed approaches have been extensively validated in benchmarking tasks, in simulation, and with real robots. On top of that, and due to the limitations inherent to those (e.g., number of robots, scenario dimensions, real-world constraints), this Thesis further contributes to the state-of-the-art by proposing a macroscopic model able to capture the RDPSO dynamics and, to some extent, analytically estimate the collective performance of robots under a certain task. It is the author’s expectation that this Ph.D. Thesis may shed some light into bridging the reality gap inherent to the applicability of swarm strategies to real-world scenarios, and in particular to SaR operations.FCT - SFRH/BD /73382/201