Analysis, evaluation and improvement of RT-WMP for real-time and QoS wireless communication: Applications in confined environments

En los ultimos años, la innovación tecnológica, la característica de flexibilidad y el rápido despligue de las redes inalámbricas, han favorecido la difusión de la redes móviles ad-hoc (MANETs), capaces de ofrecer servicios para tareas específicas entre nodos móviles. Los aspectos relacionados al dinamismo de la topología móvil y el acceso a un medio compartido por naturaleza hacen que sea preciso enfrentarse a clases de problemas distintos de los relacionados con la redes cableadas, atrayendo de este modo el interés de la comunidad científica. Las redes ad-hoc suelen soportar tráfico con garantía de servicio mínimo y la mayor parte de las propuestas presentes en literatura tratan de dar garantías de ancho de banda o minimizar el retardo de los mensajes. Sin embargo hay situaciones en las que estas garantías no son suficientes. Este es el caso de los sistemas que requieren garantías mas fuertes en la entrega de los mensajes, como es el caso de los sistemas de tiempo real donde la pérdida o el retraso de un sólo mensaje puede provocar problemas graves. Otras aplicaciones como la videoconferencia, cada vez más extendidas, implican un tráfico de datos con requisitos diferentes, como la calidad de servicio (QoS). Los requisitos de tiempo real y de QoS añaden nuevos retos al ya exigente servicio de comunicación inalámbrica entre estaciones móviles de una MANET. Además, hay aplicaciones en las que hay que tener en cuenta algo más que el simple encaminamiento de los mensajes. Este es el caso de aplicaciones en entornos subterráneos, donde el conocimiento de la evolución de propagación de la señal entre los diferentes nodos puede ser útil para mejorar la calidad de servicio y mantener la conectividad en cada momento. A pesar de ésto, dentro del amplio abanicos de propuestas presente en la literatura, existen un conjunto de limitaciones que van de el mero uso de protocolos simulados a propuestas que no tienen en cuenta entornos no convencionales o que resultan aisladas desde el punto de vista de la integración en sistemas complejos. En esta tesis doctoral, se propone un estudio completo sobre un plataforma inalámbrica de tiempo real, utilizando el protocolo RT-WMP capaz de gestionar trafíco multimedia al mismo tiempo y adaptado al entorno de trabajo. Se propone una extensión para el soporte a los datos con calidad de servicio sin limitar las caractaristícas temporales del protocolo básico. Y con el fin de tener en cuenta el efecto de la propagación de la señal, se caracteriza el entorno por medio de un conjunto de restricciones de conectividad. La solución ha sido desarrollada y su validez ha sido demostrada extensamente en aplicaciones reales en entornos subterráneos, en redes malladas y aplicaciones robóticas

Ground robotics in tunnels: Keys and lessons learned after 10 years of research and experiments

The work reported in this article describes the research advances and the lessons learned by the Robotics, Perception and Real-Time group over a decade of research in the field of ground robotics in confined environments. This study has primarily focused on localization, navigation, and communications in tunnel-like environments. As will be discussed, this type of environment presents several special characteristics that often make well-established techniques fail. The aim is to share, in an open way, the experience, errors, and successes of this group with the robotics community so that those that work in such environments can avoid (some of) the errors made. At the very least, these findings can be readily taken into account when designing a solution, without needing to sift through the technical details found in the papers cited within this text

Wireless Real-Time Communication in Tunnel-like Environments using Wireless Mesh Networks: The WICKPro Protocol

En los últimos años, las redes inalámbricas se están utilizando cada vez más en entornos industriales debido a sus ventajas respecto a redes cableadas: menor coste de instalación, soporte de movilidad, instalación en lugares donde los cables pueden ser problemáticos y mayor facilidad de reconfiguración. Estas redes inalámbricas normalmente deben proporcionar comunicación en tiempo real para satisfacer los requerimientos de las aplicaciones. Podemos encontrar ejemplos de comunicación en tiempo real con redes inalámbricas para entornos industriales en el campo de la automatización industrial y en el control de procesos, donde redes inalámbricas de radiofrecuencia han sido utilizadas para posibilitar comunicación en tiempo real con un despliegue sencillo. Asimismo, la industria también está interesada en comunicaciones en tiempo real en entornos subterráneos, puesto que existen diversas actividades que se llevan a cabo en escenarios tales como túneles y minas, incluyendo operaciones de minería, vigilancia, intervención y rescate. Las redes inalámbricas malladas (Wireless Mesh Networks, WMNs) representan una solución prometedora para conseguir comunicaci ón en tiempo real en entornos inalámbricos, dado que proporcionan una red troncal inalámbrica formada por encaminadores (routers) que es utilizada por terminales móviles. Sin embargo, las WMNs también presentan algunos retos: la naturaleza multisalto de estas redes causa interferencias entre flujos e interferencias de un flujo consigo mismo, además de que la propagación inalámbrica sufre shadowing y propagación multicamino. El estándar IEEE 802.11 ha sido ampliamente utilizado en redes WMNs debido a su bajo coste y la operación en bandas frecuenciales sin licencia. El problema es que su protocolo de acceso al medio (Medium Access Control, MAC) no es determinista y que sus comunicaciones sufren los problemas del terminal oculto y expuesto. Esta tesis doctoral se centra en el soporte de comunicaciones en tiempo real en entornos tipo túnel utilizando redes WMNs. Con este objetivo, desarrollamos un protocolo MAC y de nivel de red denominado WIreless Chain networK Protocol (WICKPro) que funciona sobre IEEE 802.11. Más concretamente, en este trabajo diseñamos dos versiones de este protocolo para proporcionar soporte de tráfico de tiempo real firme (Firm Real-Time, FRT) y de tiempo real no estricto (Soft Real-Time, SRT): FRT-WICKPro y SRT-WICKPro. Asimismo, proponemos un algoritmo de hand-off conocido como Double-Threshold Hand-off (DoTHa) para el manejo de la movilidad en SRT-WICKPro WICKPro utiliza un esquema de paso de testigo para solventar las interferencias entre flujos y de un flujo consigo mismo, así como los problemas del terminal oculto y expuesto, dado que este esquema no permite que dos nodos transmitan al mismo tiempo. Esta solución es razonable para redes pequeñas donde el re uso espacial es imposible o limitado. Para tratar la naturaleza no determinista de IEEE 802.11, combinamos el esquema de paso de testigo con una planificación cíclica global. Como es habitual en planificación cíclica, el hiperperiodo es dividido en un conjunto de ciclos secundarios. FRT-WICKPro inicia el paso de testigo de forma síncrona para satisfacer estrictamente dichos ciclos secundarios, mientras que SRT-WICKPro implementa un paso de testigo asíncrono y permite sobrepasar los ciclos secundarios, por lo que desacopla los ciclos secundarios reales de los te_oricos. Finalmente, DoTHa lidia con el shadowing y la propagación multicamino. Para abordar el shadowing, DoTHa permite llevar a cabo el proceso de hand-off en la región conectada y en la región de transición de un enlace, mientras que la propagación multicamino es ignorada para el proceso de hand-off porque la potencia recibida es promediada. Nuestras propuestas fueron validadas en experimentos de laboratorio y de campo, así como en simulación. Como un estudio de caso, llevamos a cabo la teleoperación de un robot móvil en dos entornos confinados: los pasillos de un edificio y el túnel del Somport. El túnel del Somport es un antiguo túnel ferroviario fuera de servicio que conecta España y Francia por los Pirineos Centrales. Aunque los robots autónomos son cada vez más importantes, la tecnología no está suficientemente madura para manejar entornos con alto dinamismo como sistemas de fabricación reconfigurables, o para realizar decisiones de vida o muerte, por ejemplo después de un desastre con contaminación radiactiva. Las aplicaciones que pueden beneficiarse de la teleoperación de robots móviles incluyen la monitorización en tiempo real y el uso de maquinaria robotizada, por ejemplo camiones dumper y máquinas tuneladoras, que podrían ser operadas remotamente para evitar poner en peligro vidas humanas.Industrial applications have been shifting towards wireless networks in recent years because they present several advantages compared with their wired counterparts: lower deployment cost, mobility support, installation in places where cables may be problematic, and easier reconfiguration. These industrial wireless networks usually must provide real-time communication to meet application requirements. Examples of wireless real-time communication for industrial applications can be found in factory automation and process control, where Radio Frequency wireless communication technologies have been employed to support flexible real-time communication with simple deployment. Likewise, industry is also interested in real-time communication in underground environments, since there are several activities that are carried out in scenarios such as tunnels and mines, including mining, surveillance, intervention, and rescue operations. Wireless Mesh Networks (WMNs) are promising enablers to achieve wireless real-time communication because they provide a wireless backbone comprised by dedicated routers that is utilized by mobile terminals. However, WMNs also present several challenges: wireless multi-hopping causes inter-flow and intra-flow interferences, and wireless propagation suffers shadowing and multi-path fading. The IEEE 802.11 standard has been widely used in WMNs due to its low cost and the operation in unlicensed frequency bands. The downside is that its Medium Access Control (MAC) protocol is non-deterministic, and that its communications suffer from the hidden and exposed terminal problems. This PhD thesis focuses on real-time communication in tunnel-like environments by using WMNs. Particularly, we develop a MAC and network protocol on top of the IEEE 802.11 standard to provide real-time capabilities, so-called WIreless Chain networK Protocol (WICKPro). Two WICKPro versions are designed to provide Firm Real-Time (FRT) or Soft Real-Time (SRT) traffic support: FRT-WICKPro and SRT-WICKPro. We also propose a hand-off algorithm dubbed Double-Threshold Hand-off (DoTHa) to manage mobility in SRT-WICKPro. WICKPro employs a token-passing scheme to solve the inter-flow and intra-flow interferences as well as the hidden and exposed terminal problems, since this scheme does not allow two nodes to transmit at the same time. This is a reasonable solution for small-scale networks where spatial reuse is impossible or limited. The non-deterministic nature of IEEE 802.11 is faced by combining the token-passing mechanism with a polling approach based on a global cyclic packet schedule. As usual in cyclic scheduling, the hyper-period is divided into minor cycles. FRT-WICKPro triggers the token synchronously and fulfills strictly minor cycles, whereas SRT-WICKPro carries out asynchronous token-passing and lets minor cycles be overrun, thereby decoupling the theoretic and the actual minor cycles. Finally, DoTHa deals with shadowing and multi-path fading. Shadowing is addressed by providing the opportunity of triggering hand-off in the connected and transitional regions of a link, while multi-path fading is neglected for hand-off purposes by smoothing the received signal power. We tested our proposals in laboratory and field experiments, as well as in simulation. As a case study, we carried out the tele-operation of a mobile robot within two confined environments: the corridors of a building and the Somport tunnel. The Somport tunnel is an old out-of-service railway tunnel that connects Spain and France through the Central Pyrenees. Although autonomous robots are becoming more and more important, technology is not mature enough to manage highly dynamic environments such as reconfigurable manufacturing systems, or to make life-and-death decisions, e.g., after a disaster with radioactivity contamination. Applications that can benefit from mobile robot tele-operation include real-time monitoring and the use of robotized machinery, for example, dumper trucks and tunneling machines, which could be remotely operated to avoid endangering human lives

Propagation, Localization and Navigation in Tunnel-like Environments

La robótica de servicio, entendida como aquella destinada al uso de uno o varios robots con fines de, por ejemplo, vigilancia, rescate e inspecciones, ha ido tomando cada vez más relevancia en los últimos años. Debido a los grandes avances en las distintas áreas de la robótica, los robots han sido capaces de ejecutar satisfactoriamente tareas que resultan peligrosas o incluso imposibles para los humanos, en diversos entornos. Entre ellos, los entornos confinados como túneles, minas y tuberías, han atraído la atención en aplicaciones relacionadas con transporte ferroviario, redes vehiculares, búsqueda y rescate, y vigilancia, tanto en el ámbito civil como militar. En muchas tareas, la utilización de varios robots resulta más provechoso que utilizar sólo uno. Para cooperar, los robots deben intercambiar información sobre el entorno y su propio estado, por lo que la comunicación entre ellos resulta crucial. Debido a la imposibilidad de utilizar redes cableadas entre robots móviles, se despliegan redes inalámbricas. Para determinar la calidad de señal entre dos robots, inicialmente se utilizaban modelos de propagación basados únicamente en la distancia entre ellos. Sin embargo, estas predicciones sólo resultan útiles en exteriores y sin la presencia de obstáculos, que sólo componen una pequeña parte de los escenarios de la robótica de servicio. Mas aún, la naturaleza altamente multi-trayecto de la propagación electromagnética en túneles hace que éstos actúen como guías de onda para cierto rango de frecuencias, extendiendo considerablemente el alcance de comunicación en comparación con entornos exteriores. Sin embargo, la señal se ve afectada con profundos desvanecimientos (llamados fadings en inglés). Esto los convierte en un reto para la robótica que considera la comunicación entre robots como fundamental. Además, la naturaleza hostil de estos entornos, así como también la falta de características visuales y estructurales, dificultan la localización en estos escenarios, cuestión que resulta fundamental para ejecutar con éxito una tarea con un robot. Los métodos de localización utilizados en interiores, como aquellos basados en SLAM visual, resultan imprecisos por la falta de características distintivas para cámaras o lásers, mientras que los sensores utilizados en exteriores, como el GPS, no funcionan dentro de túneles o tuberías. En esta tesis abordamos problemas fundamentales para la robótica con el fin de proporcionar herramientas necesarias para la exploración con robots en entornos tipo túnel, manteniendo la conectividad de la red de comunicaciones formada por varios robots y una estación base. Para ello, primeramente caracterizamos, en términos de propagación, los dos escenarios tipo túnel más comunes: un túnel de hormigón y una tubería metálica. Hacemos énfasis en el fenómeno de los fadings, ya que son el problema más importante a considerar para mantener la comunicación. Posteriormente presentamos una estrategia de navegación para desplegar un equipo de robots en un túnel, lidiando con los fadings para mantener la conectividad de la red formada por los robots. Esta estrategia ha sido validada a través de numerosos experimentos realizados en un túnel real, el túnel de Somport. Luego, abordamos el problema de la localización, proponiendo e implementando una técnica que permite estimar la posición de un robot dentro de una tubería, basada en la periodicidad de los fadings. El método es validado a través de experimentos reales en tuberías de pequeña y grandes dimensiones. Finalmente, proponemos esquemas de diversidad espacial, de forma que se facilita la navegación mientras se mejora la localización.Deploying a team of robots for search and rescue, inspection, or surveillance, has increasingly gained attention in the last years. As a result of the advances in several areas of robotics, robots have been able to successfully execute tasks that are hazardous or even impossible for humans in a variety of scenarios, such as outdoors, indoors, or even underground. Among these scenarios, tunnel-like environments (such as tunnels, mines, or pipes) have attracted attention for train applications, vehicular networks, search and rescue, and even service and surveillance missions in both military and civilian contexts. In most of the tasks, utilizing a multi-robot team yields better results than a singlerobot system, as it makes the system more robust while reducing the time required to complete tasks. In order to cooperate, robots must exchange information about their current state and the surrounding environment, making communication between them a crucial task. However, due to the mobile nature of robots used for exploration, a wired architecture is not possible nor convenient. Instead, a wireless network is often deployed. Wireless propagation in tunnel-like environments, characterized for the presence of strong fading phenomena, differs from regular indoor and outdoor scenarios, posing multiple challenges for communication-aware robotics. In addition, accurate localization is a problem in environments such as tunnels or pipes. These environments generally lack distinctive visual and/or structural features and are longer than they are wide in shape. Standard indoor localization techniques do not perform well in pipelines or tunnels given the lack of exploitable features, while outdoor techniques (GPS in particular) do not work in these scenarios. In this thesis, we address basic robotics-related problems in order to provide some tools necessary for robotics exploration in tunnel-like scenarios under connectivity constraints. In the first part, we characterize, in terms of propagation, two of the most common tunnel-like environments: a pipe and a tunnel. We emphasize the spatial-fadings phenomena, as it is one of the most relevant issues to deal with, in a communications context. Secondly, we present a navigation strategy to deploy a team of robots for tunnel exploration, in particular maintaining network connectivity in the presence of these fadings. Several experiments conducted in a tunnel allow us to validate the connectivity maintenance of the system. Next, we address the localization problem and propose a technique that uses the periodicity of the fadings to estimate the position of the robots from the base station. The method is validated in small-scale and large-scale pipes. Finally, we propose spatial diversity schemes in order to ease the navigation while improving the localization

Mapping and Semantic Perception for Service Robotics

Para realizar una tarea, los robots deben ser capaces de ubicarse en el entorno. Si un robot no sabe dónde se encuentra, es imposible que sea capaz de desplazarse para alcanzar el objetivo de su tarea. La localización y construcción de mapas simultánea, llamado SLAM, es un problema estudiado en la literatura que ofrece una solución a este problema. El objetivo de esta tesis es desarrollar técnicas que permitan a un robot comprender el entorno mediante la incorporación de información semántica. Esta información también proporcionará una mejora en la localización y navegación de las plataformas robóticas. Además, también demostramos cómo un robot con capacidades limitadas puede construir de forma fiable y eficiente los mapas semánticos necesarios para realizar sus tareas cotidianas.El sistema de construcción de mapas presentado tiene las siguientes características: En el lado de la construcción de mapas proponemos la externalización de cálculos costosos a un servidor en nube. Además, proponemos métodos para registrar información semántica relevante con respecto a los mapas geométricos estimados. En cuanto a la reutilización de los mapas construidos, proponemos un método que combina la construcción de mapas con la navegación de un robot para explorar mejor un entorno y disponer de un mapa semántico con los objetos relevantes para una misión determinada.En primer lugar, desarrollamos un algoritmo semántico de SLAM visual que se fusiona los puntos estimados en el mapa, carentes de sentido, con objetos conocidos. Utilizamos un sistema monocular de SLAM basado en un EKF (Filtro Extendido de Kalman) centrado principalmente en la construcción de mapas geométricos compuestos únicamente por puntos o bordes; pero sin ningún significado o contenido semántico asociado. El mapa no anotado se construye utilizando sólo la información extraída de una secuencia de imágenes monoculares. La parte semántica o anotada del mapa -los objetos- se estiman utilizando la información de la secuencia de imágenes y los modelos de objetos precalculados. Como segundo paso, mejoramos el método de SLAM presentado anteriormente mediante el diseño y la implementación de un método distribuido. La optimización de mapas y el almacenamiento se realiza como un servicio en la nube, mientras que el cliente con poca necesidad de computo, se ejecuta en un equipo local ubicado en el robot y realiza el cálculo de la trayectoria de la cámara. Los ordenadores con los que está equipado el robot se liberan de la mayor parte de los cálculos y el único requisito adicional es una conexión a Internet.El siguiente paso es explotar la información semántica que somos capaces de generar para ver cómo mejorar la navegación de un robot. La contribución en esta tesis se centra en la detección 3D y en el diseño e implementación de un sistema de construcción de mapas semántico.A continuación, diseñamos e implementamos un sistema de SLAM visual capaz de funcionar con robustez en entornos poblados debido a que los robots de servicio trabajan en espacios compartidos con personas. El sistema presentado es capaz de enmascarar las zonas de imagen ocupadas por las personas, lo que aumenta la robustez, la reubicación, la precisión y la reutilización del mapa geométrico. Además, calcula la trayectoria completa de cada persona detectada con respecto al mapa global de la escena, independientemente de la ubicación de la cámara cuando la persona fue detectada.Por último, centramos nuestra investigación en aplicaciones de rescate y seguridad. Desplegamos un equipo de robots en entornos que plantean múltiples retos que implican la planificación de tareas, la planificación del movimiento, la localización y construcción de mapas, la navegación segura, la coordinación y las comunicaciones entre todos los robots. La arquitectura propuesta integra todas las funcionalidades mencionadas, asi como varios aspectos de investigación novedosos para lograr una exploración real, como son: localización basada en características semánticas-topológicas, planificación de despliegue en términos de las características semánticas aprendidas y reconocidas, y construcción de mapas.In order to perform a task, robots need to be able to locate themselves in the environment. If a robot does not know where it is, it is impossible for it to move, reach its goal and complete the task. Simultaneous Localization and Mapping, known as SLAM, is a problem extensively studied in the literature for enabling robots to locate themselves in unknown environments. The goal of this thesis is to develop and describe techniques to allow a service robot to understand the environment by incorporating semantic information. This information will also provide an improvement in the localization and navigation of robotic platforms. In addition, we also demonstrate how a simple robot can reliably and efficiently build the semantic maps needed to perform its quotidian tasks. The mapping system as built has the following features. On the map building side we propose the externalization of expensive computations to a cloud server. Additionally, we propose methods to register relevant semantic information with respect to the estimated geometrical maps. Regarding the reuse of the maps built, we propose a method that combines map building with robot navigation to better explore a room in order to obtain a semantic map with the relevant objects for a given mission. Firstly, we develop a semantic Visual SLAM algorithm that merges traditional with known objects in the estimated map. We use a monocular EKF (Extended Kalman Filter) SLAM system that has mainly been focused on producing geometric maps composed simply of points or edges but without any associated meaning or semantic content. The non-annotated map is built using only the information extracted from an image sequence. The semantic or annotated parts of the map –the objects– are estimated using the information in the image sequence and the precomputed object models. As a second step we improve the EKF SLAM presented previously by designing and implementing a visual SLAM system based on a distributed framework. The expensive map optimization and storage is allocated as a service in the Cloud, while a light camera tracking client runs on a local computer. The robot’s onboard computers are freed from most of the computation, the only extra requirement being an internet connection. The next step is to exploit the semantic information that we are able to generate to see how to improve the navigation of a robot. The contribution of this thesis is focused on 3D sensing which we use to design and implement a semantic mapping system. We then design and implement a visual SLAM system able to perform robustly in populated environments due to service robots work in environments where people are present. The system is able to mask the image regions occupied by people out of the rigid SLAM pipeline, which boosts the robustness, the relocation, the accuracy and the reusability of the geometrical map. In addition, it estimates the full trajectory of each detected person with respect to the scene global map, irrespective of the location of the moving camera at the point when the people were imaged. Finally, we focus our research on rescue and security applications. The deployment of a multirobot team in confined environments poses multiple challenges that involve task planning, motion planning, localization and mapping, safe navigation, coordination and communications among all the robots. The architecture integrates, jointly with all the above-mentioned functionalities, several novel features to achieve real exploration: localization based on semantic-topological features, deployment planning in terms of the semantic features learned and recognized, and map building.<br /

Exploiting multimodality and structure in world representations

An essential aim of artificial intelligence research is to design agents that will eventually cooperate with humans within the real world. To this end, embodied learning is emerging as one of the most important efforts contributed by the machine learning community towards this goal. Recently developing sub-fields concern various aspects of such systems---visual reasoning, language representations, causal mechanisms, robustness to out-of-distribution inputs, to name only a few. In particular, multimodal learning and language grounding are vital to achieving a strong understanding of the real world. Humans build internal representations via interacting with their environment, learning complex associations between visual, auditory and linguistic concepts. Since the world abounds with structure, graph-based encodings are also likely to be incorporated in reasoning and decision-making modules. Furthermore, these relational representations are rather symbolic in nature---providing advantages over other formats, such as raw pixels---and can encode various types of links (temporal, causal, spatial) which can be essential for understanding and acting in the real world. This thesis presents three research works that study and develop likely aspects of future intelligent agents. The first contribution centers on vision-and-language learning, introducing a challenging embodied task that shifts the focus of an existing one to the visual reasoning problem. By extending popular visual question answering (VQA) paradigms, I also designed several models that were evaluated on the novel dataset. This produced initial performance estimates for environment understanding, through the lens of a more challenging VQA downstream task. The second work presents two ways of obtaining hierarchical representations of graph-structured data. These methods either scaled to much larger graphs than the ones processed by the best-performing method at the time, or incorporated theoretical properties via the use of topological data analysis algorithms. Both approaches competed with contemporary state-of-the-art graph classification methods, even outside social domains in the second case, where the inductive bias was PageRank-driven. Finally, the third contribution delves further into relational learning, presenting a probabilistic treatment of graph representations in complex settings such as few-shot, multi-task learning and scarce-labelled data regimes. By adding relational inductive biases to neural processes, the resulting framework can model an entire distribution of functions which generate datasets with structure. This yielded significant performance gains, especially in the aforementioned complex scenarios, with semantically-accurate uncertainty estimates that drastically improved over the neural process baseline. This type of framework may eventually contribute to developing lifelong-learning systems, due to its ability to adapt to novel tasks and distributions. The benchmark, methods and frameworks that I have devised during my doctoral studies suggest important future directions for embodied and graph representation learning research. These areas have increasingly proved their relevance to designing intelligent and collaborative agents, which we may interact with in the near future. By addressing several challenges in this problem space, my contributions therefore take a few steps towards building machine learning systems to be deployed in real-life settings.DREAM CD

Evolutionary Robot Swarms Under Real-World Constraints

Tese de doutoramento em Engenharia Electrotécnica e de Computadores, na especialidade de Automação e Robótica, apresentada ao Departamento de Engenharia Electrotécnica e de Computadores da Faculdade de Ciências e Tecnologia da Universidade de CoimbraNas últimas décadas, vários cientistas e engenheiros têm vindo a estudar as estratégias provenientes da natureza. Dentro das arquiteturas biológicas, as sociedades que vivem em enxames revelam que agentes simplistas, tais como formigas ou pássaros, são capazes de realizar tarefas complexas usufruindo de mecanismos de cooperação. Estes sistemas abrangem todas as condições necessárias para a sobrevivência, incorporando comportamentos de cooperação, competição e adaptação. Na “batalha” sem fim em prol do progresso dos mecanismos artificiais desenvolvidos pelo homem, a ciência conseguiu simular o primeiro comportamento em enxame no final dos anos oitenta. Desde então, muitas outras áreas, entre as quais a robótica, beneficiaram de mecanismos de tolerância a falhas inerentes da inteligência coletiva de enxames. A área de investigação deste estudo incide na robótica de enxame, consistindo num domínio particular dos sistemas robóticos cooperativos que incorpora os mecanismos de inteligência coletiva de enxames na robótica. Mais especificamente, propõe-se uma solução completa de robótica de enxames a ser aplicada em contexto real. Nesta ótica, as operações de busca e salvamento foram consideradas como o caso de estudo principal devido ao nível de complexidade associado às mesmas. Tais operações ocorrem tipicamente em cenários dinâmicos de elevadas dimensões, com condições adversas que colocam em causa a aplicabilidade dos sistemas robóticos cooperativos. Este estudo centra-se nestes problemas, procurando novos desafios que não podem ser ultrapassados através da simples adaptação da literatura da especialidade em algoritmos de enxame, planeamento, controlo e técnicas de tomada de decisão. As contribuições deste trabalho sustentam-se em torno da extensão do método Particle Swarm Optimization (PSO) aplicado a sistemas robóticos cooperativos, denominado de Robotic Darwinian Particle Swarm Optimization (RDPSO). O RDPSO consiste numa arquitetura robótica de enxame distribuída que beneficia do particionamento dinâmico da população de robôs utilizando mecanismos evolucionários de exclusão social baseados na sobrevivência do mais forte de Darwin. No entanto, apesar de estar assente no caso de estudo do RDPSO, a aplicabilidade dos conceitos aqui propostos não se encontra restrita ao mesmo, visto que todos os algoritmos parametrizáveis de enxame de robôs podem beneficiar de uma abordagem idêntica. Os fundamentos em torno do RDPSO são introduzidos, focando-se na dinâmica dos robôs, nos constrangimentos introduzidos pelos obstáculos e pela comunicação, e nas suas propriedades evolucionárias. Considerando a colocação inicial dos robôs no ambiente como algo fundamental para aplicar sistemas de enxames em aplicações reais, é assim introduzida uma estratégia de colocação de robôs realista. Para tal, a população de robôs é dividida de forma hierárquica, em que são utilizadas plataformas mais robustas para colocar as plataformas de enxame no cenário de forma autónoma. Após a colocação dos robôs no cenário, é apresentada uma estratégia para permitir a criação e manutenção de uma rede de comunicação móvel ad hoc com tolerância a falhas. Esta estratégia não considera somente a distância entre robôs, mas também a qualidade do nível de sinal rádio frequência, redefinindo assim a sua aplicabilidade em cenários reais. Os aspetos anteriormente mencionados estão sujeitos a uma análise detalhada do sistema de comunicação inerente ao algoritmo, para atingir uma implementação mais escalável do RDPSO a cenários de elevada complexidade. Esta elevada complexidade inerente à dinâmica dos cenários motivaram a ultimar o desenvolvimento do RDPSO, integrando para o efeito um mecanismo adaptativo baseado em informação contextual (e.g., nível de atividade do grupo). Face a estas considerações, o presente estudo pode contribuir para expandir o estado-da-arte em robótica de enxame com algoritmos inovadores aplicados em contexto real. Neste sentido, todos os métodos propostos foram extensivamente validados e comparados com alternativas, tanto em simulação como com robôs reais. Para além disso, e dadas as limitações destes (e.g., número limitado de robôs, cenários de dimensões limitadas, constrangimentos reais limitados), este trabalho contribui ainda para um maior aprofundamento do estado-da-arte, onde se propõe um modelo macroscópico capaz de capturar a dinâmica inerente ao RDPSO e, até certo ponto, estimar analiticamente o desempenho coletivo dos robôs perante determinada tarefa. Em suma, esta investigação pode ter aplicabilidade prática ao colmatar a lacuna que se faz sentir no âmbito das estratégias de enxames de robôs em contexto real e, em particular, em cenários de busca e salvamento.Over the past decades, many scientists and engineers have been studying nature’s best and time-tested patterns and strategies. Within the existing biological architectures, swarm societies revealed that relatively unsophisticated agents with limited capabilities, such as ants or birds, were able to cooperatively accomplish complex tasks necessary for their survival. Those simplistic systems embrace all the conditions necessary to survive, thus embodying cooperative, competitive and adaptive behaviours. In the never-ending battle to advance artificial manmade mechanisms, computer scientists simulated the first swarm behaviour designed to mimic the flocking behaviour of birds in the late eighties. Ever since, many other fields, such as robotics, have benefited from the fault-tolerant mechanism inherent to swarm intelligence. The area of research presented in this Ph.D. Thesis focuses on swarm robotics, which is a particular domain of multi-robot systems (MRS) that embodies the mechanisms of swarm intelligence into robotics. More specifically, this Thesis proposes a complete swarm robotic solution that can be applied to real-world missions. Although the proposed methods do not depend on any particular application, search and rescue (SaR) operations were considered as the main case study due to their inherent level of complexity. Such operations often occur in highly dynamic and large scenarios, with harsh and faulty conditions, that pose several problems to MRS applicability. This Thesis focuses on these problems raising new challenges that cannot be handled appropriately by simple adaptation of state-of-the-art swarm algorithms, planning, control and decision-making techniques. The contributions of this Thesis revolve around an extension of the Particle Swarm Optimization (PSO) to MRS, denoted as Robotic Darwinian Particle Swarm Optimization (RDPSO). The RDPSO is a distributed swarm robotic architecture that benefits from the dynamical partitioning of the whole swarm of robots by means of an evolutionary social exclusion mechanism based on Darwin’s survival-of-the-fittest. Nevertheless, although currently applied solely to the RDPSO case study, the applicability of all concepts herein proposed is not restricted to it, since all parameterized swarm robotic algorithms may benefit from a similar approach The RDPSO is then proposed and used to devise the applicability of novel approaches. The fundamentals around the RDPSO are introduced by focusing on robots’ dynamics, obstacle avoidance, communication constraints and its evolutionary properties. Afterwards, taking the initial deployment of robots within the environment as a basis for applying swarm robotics systems into real-world applications, the development of a realistic deployment strategy is proposed. For that end, the population of robots is hierarchically divided, wherein larger support platforms autonomously deploy smaller exploring platforms in the scenario, while considering communication constraints and obstacles. After the deployment, a way of ensuring a fault-tolerant multi-hop mobile ad hoc communication network (MANET) is introduced to explicitly exchange information needed in a collaborative realworld task execution. Such strategy not only considers the maximum communication range between robots, but also the minimum signal quality, thus refining the applicability to real-world context. This is naturally followed by a deep analysis of the RDPSO communication system, describing the dynamics of the communication data packet structure shared between teammates. Such procedure is a first step to achieving a more scalable implementation by optimizing the communication procedure between robots. The highly dynamic characteristics of real-world applications motivated us to ultimate the RDPSO development with an adaptive strategy based on a set of context-based evaluation metrics. This thesis contributes to the state-of-the-art in swarm robotics with novel algorithms for realworld applications. All of the proposed approaches have been extensively validated in benchmarking tasks, in simulation, and with real robots. On top of that, and due to the limitations inherent to those (e.g., number of robots, scenario dimensions, real-world constraints), this Thesis further contributes to the state-of-the-art by proposing a macroscopic model able to capture the RDPSO dynamics and, to some extent, analytically estimate the collective performance of robots under a certain task. It is the author’s expectation that this Ph.D. Thesis may shed some light into bridging the reality gap inherent to the applicability of swarm strategies to real-world scenarios, and in particular to SaR operations.FCT - SFRH/BD /73382/201

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors

Object Tracking

Object tracking consists in estimation of trajectory of moving objects in the sequence of images. Automation of the computer object tracking is a difficult task. Dynamics of multiple parameters changes representing features and motion of the objects, and temporary partial or full occlusion of the tracked objects have to be considered. This monograph presents the development of object tracking algorithms, methods and systems. Both, state of the art of object tracking methods and also the new trends in research are described in this book. Fourteen chapters are split into two sections. Section 1 presents new theoretical ideas whereas Section 2 presents real-life applications. Despite the variety of topics contained in this monograph it constitutes a consisted knowledge in the field of computer object tracking. The intention of editor was to follow up the very quick progress in the developing of methods as well as extension of the application