Distributed consensus in multi-robot systems with visual perception

La idea de equipos de robots actuando con autonomía y de manera cooperativa está cada día más cerca de convertirse en realidad. Los sistemas multi robot pueden ejecutar tareas de gran complejidad con mayor robustez y en menos tiempo que un robot trabajando solo. Por otra parte, la coordinación de un equipo de robots introduce complicaciones que los ingenieros encargados de diseñar estos sistemas deben afrontar. Conseguir que la percepción del entorno sea consistente en todos los robots es uno de los aspectos más importantes requeridos en cualquier tarea cooperativa, lo que implica que las observaciones de cada robot del equipo deben ser transmitidas a todos los otros miembros. Cuando dos o más robots poseen información común del entorno, el equipo debe alcanzar un consenso usando toda la información disponible. Esto se debe hacer considerando las limitaciones de cada robot, teniendo en cuenta que no todos los robots se pueden comunicar unos con otros. Con este objetivo, se aborda la tarea de diseñar algoritmos distribuidos que consigan que un equipo de robots llegue a un consenso acerca de la información percibida por todos los miembros. Específicamente, nos centramos en resolver este problema cuando los robots usan la visión como sensor para percibir el entorno. Las cámaras convencionales son muy útiles a la hora de ejecutar tareas como la navegación y la construcción de mapas, esenciales en el ámbito de la robótica, gracias a la gran cantidad de información que contiene cada imagen. Sin embargo, el uso de estos sensores en un marco distribuido introduce una gran cantidad de complicaciones adicionales que deben ser abordadas si se quiere cumplir el objetivo propuesto. En esta Tesis presentamos un estudio profundo de los algoritmos distribuidos de consenso y cómo estos pueden ser usados por un equipo de robots equipados con cámaras convencionales, resolviendo los aspectos más importantes relacionados con el uso de estos sensores. En la primera parte de la Tesis nos centramos en encontrar correspondencias globales entre las observaciones de todos los robots. De esta manera, los robots son capaces de detectar que observaciones deben ser combinadas para el cálculo del consenso. También lidiamos con el problema de la robustez y la detección distribuida de espurios durante el cálculo del consenso. Para contrarrestar el incremento del tamaño de los mensajes intercambiados por los robots en las etapas anteriores, usamos las propiedades de los polinomios de Chebyshev, reduciendo el número de iteraciones que se requieren para alcanzar el consenso. En la segunda parte de la Tesis, centramos nuestra atención en los problemas de crear un mapa y controlar el movimiento del equipo de robots. Presentamos soluciones para alcanzar un consenso en estos escenarios mediante el uso de técnicas de visión por computador ampliamente conocidas. El uso de algoritmos de estructura y movimiento nos permite obviar restricciones tales como que los robots tengan que observarse unos a otros directamente durante el control o la necesidad de especificar un marco de referencia común. Adicionalmente, nuestros algoritmos tienen un comportamiento robusto cuando la calibración de las cámaras no se conoce. Finalmente, la evaluación de las propuestas se realiza utilizando un data set de un entorno urbano y robots reales con restricciones de movimiento no holónomas. Todos los algoritmos que se presentan en esta Tesis han sido diseñados para ser ejecutados de manera distribuida. En la Tesis demostramos de manera teórica las principales propiedades de los algoritmos que se proponen y evaluamos la calidad de los mismos con datos simulados e imágenes reales. En resumen, las principales contribuciones de esta Tesis son: • Un conjunto de algoritmos distribuidos que permiten a un equipo de robots equipados con cámaras convencionales alcanzar un consenso acerca de la información que perciben. En particular, proponemos tres algoritmos distribuidos con el objetivo de resolver los problemas de encontrar correspondencias globales entre la información de todos los robots, detectar y descartar información espuria, y reducir el número de veces que los robots tienen que comunicarse entre ellos antes de alcanzar el consenso. • La combinación de técnicas de consenso distribuido y estructura y movimiento en tareas de control y percepción. Se ha diseñado un algoritmo para construir un mapa topológico de manera cooperativa usando planos como características del mapa y restricciones de homografía como elementos para relacionar las observaciones de los robots. También se ha propuesto una ley de control distribuida utilizando la geometría epipolar con el objetivo de hacer que el equipo de robots alcance una orientación común sin la necesidad de observarse directamente unos a otros

RANSAC for Robotic Applications: A Survey

Random Sample Consensus, most commonly abbreviated as RANSAC, is a robust estimation method for the parameters of a model contaminated by a sizable percentage of outliers. In its simplest form, the process starts with a sampling of the minimum data needed to perform an estimation, followed by an evaluation of its adequacy, and further repetitions of this process until some stopping criterion is met. Multiple variants have been proposed in which this workflow is modified, typically tweaking one or several of these steps for improvements in computing time or the quality of the estimation of the parameters. RANSAC is widely applied in the field of robotics, for example, for finding geometric shapes (planes, cylinders, spheres, etc.) in cloud points or for estimating the best transformation between different camera views. In this paper, we present a review of the current state of the art of RANSAC family methods with a special interest in applications in robotics.This work has been partially funded by the Basque Government, Spain, under Research Teams Grant number IT1427-22 and under ELKARTEK LANVERSO Grant number KK-2022/00065; the Spanish Ministry of Science (MCIU), the State Research Agency (AEI), the European Regional Development Fund (FEDER), under Grant number PID2021-122402OB-C21 (MCIU/AEI/FEDER, UE); and the Spanish Ministry of Science, Innovation and Universities, under Grant FPU18/04737

Distributed scene reconstruction from multiple mobile platforms

Recent research on mobile robotics has produced new designs that provide house-hold robots with omnidirectional motion. The image sensor embedded in these devices motivates the application of 3D vision techniques on them for navigation and mapping purposes. In addition to this, distributed cheapsensing systems acting as unitary entity have recently been discovered as an efficient alternative to expensive mobile equipment. In this work we present an implementation of a visual reconstruction method, structure from motion (SfM), on a low-budget, omnidirectional mobile platform, and extend this method to distributed 3D scene reconstruction with several instances of such a platform. Our approach overcomes the challenges yielded by the plaform. The unprecedented levels of noise produced by the image compression typical of the platform is processed by our feature filtering methods, which ensure suitable feature matching populations for epipolar geometry estimation by means of a strict quality-based feature selection. The robust pose estimation algorithms implemented, along with a novel feature tracking system, enable our incremental SfM approach to novelly deal with ill-conditioned inter-image configurations provoked by the omnidirectional motion. The feature tracking system developed efficiently manages the feature scarcity produced by noise and outputs quality feature tracks, which allow robust 3D mapping of a given scene even if - due to noise - their length is shorter than what it is usually assumed for performing stable 3D reconstructions. The distributed reconstruction from multiple instances of SfM is attained by applying loop-closing techniques. Our multiple reconstruction system merges individual 3D structures and resolves the global scale problem with minimal overlaps, whereas in the literature 3D mapping is obtained by overlapping stretches of sequences. The performance of this system is demonstrated in the 2-session case. The management of noise, the stability against ill-configurations and the robustness of our SfM system is validated on a number of experiments and compared with state-of-the-art approaches. Possible future research areas are also discussed

Euclidean reconstruction of natural underwater scenes using optic imagery sequence

The development of maritime applications require monitoring, studying and preserving of detailed and close observation on the underwater seafloor and objects. Stereo vision offers advanced technologies to build 3D models from 2D still overlapping images in a relatively inexpensive way. However, while image stereo matching is a necessary step in 3D reconstruction procedure, even the most robust dense matching techniques are not guaranteed to work for underwater images due to the challenging aquatic environment. In this thesis, in addition to a detailed introduction and research on the key components of building 3D models from optic images, a robust modified quasi-dense matching algorithm based on correspondence propagation and adaptive least square matching for underwater images is proposed and applied to some typical underwater image datasets. The experiments demonstrate the robustness and good performance of the proposed matching approach

Distributed Robotic Vision for Calibration, Localisation, and Mapping

This dissertation explores distributed algorithms for calibration, localisation, and mapping in the context of a multi-robot network equipped with cameras and onboard processing, comparing against centralised alternatives where all data is transmitted to a singular external node on which processing occurs. With the rise of large-scale camera networks, and as low-cost on-board processing becomes increasingly feasible in robotics networks, distributed algorithms are becoming important for robustness and scalability. Standard solutions to multi-camera computer vision require the data from all nodes to be processed at a central node which represents a significant single point of failure and incurs infeasible communication costs. Distributed solutions solve these issues by spreading the work over the entire network, operating only on local calculations and direct communication with nearby neighbours. This research considers a framework for a distributed robotic vision platform for calibration, localisation, mapping tasks where three main stages are identified: an initialisation stage where calibration and localisation are performed in a distributed manner, a local tracking stage where visual odometry is performed without inter-robot communication, and a global mapping stage where global alignment and optimisation strategies are applied. In consideration of this framework, this research investigates how algorithms can be developed to produce fundamentally distributed solutions, designed to minimise computational complexity whilst maintaining excellent performance, and designed to operate effectively in the long term. Therefore, three primary objectives are sought aligning with these three stages

Matching As Color Imaging: Thermal Feature Detection

Feature detection and extraction is considered to be one of the most important aspects when it comes to any computer vision application, especially the autonomous driving field that is highly dependent on it. Thermal imaging is less explored in the field of autonomous driving mainly due to the high cost of the cameras and inferior techniques available for detection. Due to advances in technology the former is not a major limitation and there lies tremendous scope for improvement in the latter. Autonomous driving relies heavily on multiple and sometimes redundant sensors, for which thermal sensors are a preferred addition. Thermal sensors being completely dependent on the infrared radiation emitted are able to frame and recognize objects even in the complete absence of light. However, detecting features persistently through subsequent frames is a difficult task due to the lack of textures in thermal images. Motivated by this challenge, we propose a Triplet based Siamese Convolutional Neural Network for feature detection and extraction for any given thermal image. Our architecture is able to detect larger number of good feature points on thermal images than other best performed feature detection algorithms with superb matching performance based on our extracted descriptors. To demonstrate our aforementioned claim, we compare the performance of the proposed CNN scheme with traditional as well as state-of-the-art feature detection and extraction schemes. Future work involves extending the pipeline for motion tracking, SLAM, SFM and many other applications