Contribuciones al uso de marcadores para Navegación Autónoma y Realidad Aumentada

Square planar markers are a widely used tools for localization and tracking due to their low cost and high performance. Many applications in Robotics, Unmanned Vehicles and Augmented Reality employ these markers for camera pose estimation with high accuracy. Nevertheless, marker-based systems are affected by several factors that limit their performance. First, the marker detection process is a time-consuming task, which is intensified as the image size increases. As a consequence, the current high-resolution cameras has weakened the processing efficiency of traditional marker systems. Second, marker detection is affected by the presence of noise, blurring and occlusion. The movement of the camera produces image blurriness, generated even by small movements. Furthermore, the marker may be partially or completely occluded in the image, so that it is no longer detected. This thesis deals with the above limitations, proposing novel methodologies and strategies for successful marker detection improving both the efficiency and robustness of these systems. First, a novel multi-scale approach has been developed to speed up the marker detection process. The method takes advantage of the different resolutions at which the image is represented to predict at runtime the optimal scale for detection and identification, as well as following a corner upsampling strategy necessary for an accurate pose estimation. Second, we introduce a new marker design, Fractal Marker, which using a novel keypoint-based method achieves detection even under severe occlusion, while allowing detection over a wider range of distance than traditional markers. Finally, we propose a new marker detection strategy based on Discriminative Correlation Filters (DCF), where the marker and its corners represented in the frequency domain perform more robust and faster detections than state-ofthe- art methods, even under extreme blur conditions.Los marcadores planos cuadrados son una de las herramientas ampliamente utilizadas para la localización y el tracking debido a su bajo coste y su alto rendimiento. Muchas aplicaciones en Robótica, Vehículos no Tripulados y Realidad Aumentada emplean estos marcadores para estimar con alta precisión la posición de la cámara. Sin embargo, los sistemas basados en marcadores se ven afectados por varios factores que limitan su rendimiento. En primer lugar, el proceso de detección de marcadores es una tarea que requiere mucho tiempo y este incrementa a medida que aumenta el tamaño de la imagen. En consecuencia, las actuales cámaras de alta resolución han debilitado la eficacia del procesamiento de los sistemas de marcadores tradicionales. Por otra parte, la detección de marcadores se ve afectada por la presencia de ruido, desenfoque y oclusión. El movimiento de la cámara produce desenfoque de la imagen, generado incluso por pequeños movimientos. Además, el marcador puede aparecer en la imagen parcial o completamente ocluido, dejando de ser detectado. Esta tesis aborda las limitaciones anteriores, proponiendo metodologías y estrategias novedosas para la correcta detección de marcadores, mejorando así tanto la eficiencia como la robustez de estos sistemas. En primer lugar, se ha desarrollado un novedoso enfoque multiescala para acelerar el proceso de detección de marcadores. El método aprovecha las diferentes resoluciones en las que la imagen está representada para predecir en tiempo de ejecución la escala óptima para la detección e identificación, a la vez que sigue una estrategia de upsampling de las esquinas necesaria para estimar la pose con precisión. En segundo lugar, introducimos un nuevo diseño de marcador, Fractal Marker, que, mediante un método basado en keypoints, logra detecciones incluso en casos de oclusión extrema, al tiempo que permite la detección en un rango de distancias más amplio que los marcadores tradicionales. Por último, proponemos una nueva estrategia de detección de marcadores basada en Discriminate Correlation Filters (DCF), donde el marcador y sus esquinas representadas en el dominio de la frecuencia realizan detecciones más robustas y rápidas que los métodos de referencia, incluso bajo condiciones extremas de emborronamiento

A Factor Graph Approach to Multi-Camera Extrinsic Calibration on Legged Robots

Legged robots are becoming popular not only in research, but also in industry, where they can demonstrate their superiority over wheeled machines in a variety of applications. Either when acting as mobile manipulators or just as all-terrain ground vehicles, these machines need to precisely track the desired base and end-effector trajectories, perform Simultaneous Localization and Mapping (SLAM), and move in challenging environments, all while keeping balance. A crucial aspect for these tasks is that all onboard sensors must be properly calibrated and synchronized to provide consistent signals for all the software modules they feed. In this paper, we focus on the problem of calibrating the relative pose between a set of cameras and the base link of a quadruped robot. This pose is fundamental to successfully perform sensor fusion, state estimation, mapping, and any other task requiring visual feedback. To solve this problem, we propose an approach based on factor graphs that jointly optimizes the mutual position of the cameras and the robot base using kinematics and fiducial markers. We also quantitatively compare its performance with other state-of-the-art methods on the hydraulic quadruped robot HyQ. The proposed approach is simple, modular, and independent from external devices other than the fiducial marker.Comment: To appear on "The Third IEEE International Conference on Robotic Computing (IEEE IRC 2019)

UcoSLAM: Simultaneous Localization and Mapping by Fusion of KeyPoints and Squared Planar Markers

This paper proposes a novel approach for Simultaneous Localization and Mapping by fusing natural and artificial landmarks. Most of the SLAM approaches use natural landmarks (such as keypoints). However, they are unstable over time, repetitive in many cases or insufficient for a robust tracking (e.g. in indoor buildings). On the other hand, other approaches have employed artificial landmarks (such as squared fiducial markers) placed in the environment to help tracking and relocalization. We propose a method that integrates both approaches in order to achieve long-term robust tracking in many scenarios. Our method has been compared to the start-of-the-art methods ORB-SLAM2 and LDSO in the public dataset Kitti, Euroc-MAV, TUM and SPM, obtaining better precision, robustness and speed. Our tests also show that the combination of markers and keypoints achieves better accuracy than each one of them independently.Comment: Paper submitted to Pattern Recognitio

Multi-target Attachment for Surgical Instrument Tracking

The pose estimation of a surgical instrument is a common problem in the new needs of medical science. Many instrument tracking methods use markers with a known geometry that allows for solving the instrument pose as detected by a camera. However, marker occlusion happens, and it hinders correct pose estimation. In this work, we propose an adaptable multi-target attachment with ArUco markers to solve occlusion problems on tracking a medical instrument like an ultrasound probe or a scalpel. Our multi-target system allows for precise and redundant real-time pose estimation implemented in OpenCV. Encouraging results show that the multi-target device may prove useful in the clinical settin

sSLAM: Speeded-Up Visual SLAM Mixing Artificial Markers and Temporary Keypoints

Environment landmarks are generally employed by visual SLAM (vSLAM) methods in the form of keypoints. However, these landmarks are unstable over time because they belong to areas that tend to change, e.g., shadows or moving objects. To solve this, some other authors have proposed the combination of keypoints and artificial markers distributed in the environment so as to facilitate the tracking process in the long run. Artificial markers are special elements (similar to beacons) that can be permanently placed in the environment to facilitate tracking. In any case, these systems keep a set of keypoints that is not likely to be reused, thus unnecessarily increasing the computing time required for tracking. This paper proposes a novel visual SLAM approach that efficiently combines keypoints and artificial markers, allowing for a substantial reduction in the computing time and memory required without noticeably degrading the tracking accuracy. In the first stage, our system creates a map of the environment using both keypoints and artificial markers, but once the map is created, the keypoints are removed and only the markers are kept. Thus, our map stores only long-lasting features of the environment (i.e., the markers). Then, for localization purposes, our algorithm uses the marker information along with temporary keypoints created just in the time of tracking, which are removed after a while. Since our algorithm keeps only a small subset of recent keypoints, it is faster than the state-of-the-art vSLAM approaches. The experimental results show that our proposed sSLAM compares favorably with ORB-SLAM2, ORB-SLAM3, OpenVSLAM and UcoSLAM in terms of speed, without statistically significant differences in accuracy.This research was funded by the project PID2019-103871GB-I00 of the Spanish Ministry of Economy, Industry and Competitiveness, FEDER, Project 1380047-F UCOFEDER-2021 of Andalusia and by the European Union–NextGeneration EU for requalification of Spanish University System 2021–2023

sSLAM: Speeded-Up Visual SLAM Mixing Artificial Markers and Temporary Keypoints

Environment landmarks are generally employed by visual SLAM (vSLAM) methods in the form of keypoints. However, these landmarks are unstable over time because they belong to areas that tend to change, e.g., shadows or moving objects. To solve this, some other authors have proposed the combination of keypoints and artificial markers distributed in the environment so as to facilitate the tracking process in the long run. Artificial markers are special elements (similar to beacons) that can be permanently placed in the environment to facilitate tracking. In any case, these systems keep a set of keypoints that is not likely to be reused, thus unnecessarily increasing the computing time required for tracking. This paper proposes a novel visual SLAM approach that efficiently combines keypoints and artificial markers, allowing for a substantial reduction in the computing time and memory required without noticeably degrading the tracking accuracy. In the first stage, our system creates a map of the environment using both keypoints and artificial markers, but once the map is created, the keypoints are removed and only the markers are kept. Thus, our map stores only long-lasting features of the environment (i.e., the markers). Then, for localization purposes, our algorithm uses the marker information along with temporary keypoints created just in the time of tracking, which are removed after a while. Since our algorithm keeps only a small subset of recent keypoints, it is faster than the state-of-the-art vSLAM approaches. The experimental results show that our proposed sSLAM compares favorably with ORB-SLAM2, ORB-SLAM3, OpenVSLAM and UcoSLAM in terms of speed, without statistically significant differences in accuracy

Melhoria do alinhamento de imagens RGB-D usando marcadores fiduciais

3D reconstruction is the creation of three-dimensional models from the captured shape and appearance of real objects. It is a field that has its roots in several areas within computer vision and graphics, and has gained high importance in others, such as architecture, robotics, autonomous driving, medicine, and archaeology. Most of the current model acquisition technologies are based on LiDAR, RGB-D cameras, and image-based approaches such as visual SLAM. Despite the improvements that have been achieved, methods that rely on professional instruments and operation result in high costs, both capital and logistical. In this dissertation, we develop an optimization procedure capable of enhancing the 3D reconstructions created using a consumer level RGB-D hand-held camera, a product that is widely available, easily handled, with a familiar interface to the average smartphone user, through the utilisation of fiducial markers placed in the environment. Additionally, a tool was developed to allow the removal of said fiducial markers from the texture of the scene, as a complement to mitigate a downside of the approach taken, but that may prove useful in other contexts.A reconstrução 3D é a criação de modelos tridimensionais a partir da forma e aparência capturadas de objetos reais. É um campo que teve origem em diversos ramos da visão computacional e computação gráfica, e que ganhou grande importância em áreas como a arquitetura, robótica, condução autónoma, medicina e arqueologia. A maioria das tecnologias de aquisição de modelos atuais são baseadas em LiDAR, câmeras RGB-D e abordagens baseadas em imagens, como o SLAM visual. Apesar das melhorias que foram alcançadas, os métodos que dependem de instrumentos profissionais e da sua operação resultam em elevados custos, tanto de capital, como logísticos. Nesta dissertação foi desenvolvido um processo de otimização capaz de melhorar as reconstruções 3D criadas usando uma câmera RGB-D portátil, disponível ao nível do consumidor, de fácil manipulação e que tem uma interface familiar para o utilizador de smartphones, através da utilização de marcadores fiduciais colocados no ambiente. Além disso, uma ferramenta foi desenvolvida para permitir a remoção dos ditos marcadores fiduciais da textura da cena, como um complemento para mitigar uma desvantagem da abordagem adotada, mas que pode ser útil em outros contextos.Mestrado em Engenharia de Computadores e Telemátic

Application of Ghost-DeblurGAN to Fiducial Marker Detection

Feature extraction or localization based on the fiducial marker could fail due to motion blur in real-world robotic applications. To solve this problem, a lightweight generative adversarial network, named Ghost-DeblurGAN, for real-time motion deblurring is developed in this paper. Furthermore, on account that there is no existing deblurring benchmark for such task, a new large-scale dataset, YorkTag, is proposed that provides pairs of sharp/blurred images containing fiducial markers. With the proposed model trained and tested on YorkTag, it is demonstrated that when applied along with fiducial marker systems to motion-blurred images, Ghost-DeblurGAN improves the marker detection significantly. The datasets and codes used in this paper are available at: https://github.com/York-SDCNLab/Ghost-DeblurGAN.Comment: 6 pages, 6 figure

Vision-based Situational Graphs Generating Optimizable 3D Scene Representations

3D scene graphs offer a more efficient representation of the environment by hierarchically organizing diverse semantic entities and the topological relationships among them. Fiducial markers, on the other hand, offer a valuable mechanism for encoding comprehensive information pertaining to environments and the objects within them. In the context of Visual SLAM (VSLAM), especially when the reconstructed maps are enriched with practical semantic information, these markers have the potential to enhance the map by augmenting valuable semantic information and fostering meaningful connections among the semantic objects. In this regard, this paper exploits the potential of fiducial markers to incorporate a VSLAM framework with hierarchical representations that generates optimizable multi-layered vision-based situational graphs. The framework comprises a conventional VSLAM system with low-level feature tracking and mapping capabilities bolstered by the incorporation of a fiducial marker map. The fiducial markers aid in identifying walls and doors in the environment, subsequently establishing meaningful associations with high-level entities, including corridors and rooms. Experimental results are conducted on a real-world dataset collected using various legged robots and benchmarked against a Light Detection And Ranging (LiDAR)-based framework (S-Graphs) as the ground truth. Consequently, our framework not only excels in crafting a richer, multi-layered hierarchical map of the environment but also shows enhancement in robot pose accuracy when contrasted with state-of-the-art methodologies.Comment: 7 pages, 6 figures, 2 table