Fast Staircase Detection and Estimation using 3D Point Clouds with Multi-detection Merging for Heterogeneous Robots

Robotic systems need advanced mobility capabilities to operate in complex, three-dimensional environments designed for human use, e.g., multi-level buildings. Incorporating some level of autonomy enables robots to operate robustly, reliably, and efficiently in such complex environments, e.g., automatically ``returning home'' if communication between an operator and robot is lost during deployment. This work presents a novel method that enables mobile robots to robustly operate in multi-level environments by making it possible to autonomously locate and climb a range of different staircases. We present results wherein a wheeled robot works together with a quadrupedal system to quickly detect different staircases and reliably climb them. The performance of this novel staircase detection algorithm that is able to run on the heterogeneous platforms is compared to the current state-of-the-art detection algorithm. We show that our approach significantly increases the accuracy and speed at which detections occur.Comment: 7 pages, 8 Figures, 2 Table

Robotic Perception for Terrain-Aware Navigation in Subterranean Search and Rescue

Navigation over torturous terrain such as those in subterranean environments presents a significant challenge to field robots. The diversity of hazards, from large boulders to muddy or even partially submerged earth, eludes complete definition. The challenge is amplified if the presence and nature of these hazards must be shared among multiple agents operating in the same space. Furthermore, highly efficient mapping and robust navigation solutions are absolutely critical to operations such as semi-autonomous search and rescue. In this dissertation, I present three contributions that promote navigation and exploration in subterranean environments by autonomous ground robots. The first contribution is a novel semanticmetric grid mapping approach to maintain global situational awareness of terrain traversability and the presence of stairs. The second contribution is the integration of the proposed grid mapping method with a map-sharing framework and a terrain-aware navigation stack to facilitate collaborative exploration of subterranean environments. The third contribution extends an advanced navigation solution to enable high-speed exploration of subterranean and other unstructured environments. This work was performed in the context of the DARPA Subterranean Challenge (&ldquo;SubT&rdquo;), a series of four competition events held between August 2018 and September 2021 which tasked teams to design multi-agent robotic exploration systems dedicated to search and rescue operations [19].</p

Improving perception and locomotion capabilities of mobile robots in urban search and rescue missions

Nasazení mobilních robotů během zásahů záchranných složek je způsob, jak učinit práci záchranářů bezpečnější a efektivnější. Na roboty jsou ale při takovém použití kladeny vyšší nároky kvůli podmínkám, které při těchto událostech panují. Roboty se musejí pohybovat po nestabilních površích, ve stísněných prostorech nebo v kouři a prachu, což ztěžuje použití některých senzorů. Lokalizace, v robotice běžná úloha spočívající v určení polohy robotu vůči danému souřadnému systému, musí spolehlivě fungovat i za těchto ztížených podmínek. V této dizertační práci popisujeme vývoj lokalizačního systému pásového mobilního robotu, který je určen pro nasazení v případě zemětřesení nebo průmyslové havárie. Nejprve je předveden lokalizační systém, který vychází pouze z měření proprioceptivních senzorů a který vyvstal jako nejlepší varianta při porovnání několika možných uspořádání takového systému. Lokalizace je poté zpřesněna přidáním měření exteroceptivních senzorů, které zpomalují kumulaci nejistoty určení polohy robotu. Zvláštní pozornost je věnována možným výpadkům jednotlivých senzorických modalit, prokluzům pásů, které u tohoto typu robotů nevyhnutelně nastávají, výpočetním nárokům lokalizačního systému a rozdílným vzorkovacím frekvencím jednotlivých senzorů. Dále se věnujeme problému kinematických modelů pro přejíždění vertikálních překážek, což je další zdroj nepřesnosti při lokalizaci pásového robotu. Díky účasti na výzkumných projektech, jejichž členy byly hasičské sbory Itálie, Německa a Nizozemska, jsme měli přístup na cvičiště určená pro přípravu na zásahy během zemětřesení, průmyslových a dopravních nehod. Přesnost našeho lokalizačního systému jsme tedy testovali v podmínkách, které věrně napodobují ty skutečné. Soubory senzorických měření a referenčních poloh, které jsme vytvořili pro testování přesnosti lokalizace, jsou veřejně dostupné a považujeme je za jeden z přínosů naší práce. Tato dizertační práce má podobu souboru tří časopiseckých publikací a jednoho článku, který je v době jejího podání v recenzním řízení.eployment of mobile robots in search and rescue missions is a way to make job of human rescuers safer and more efficient. Such missions, however, require robots to be resilient to harsh conditions of natural disasters or human-inflicted accidents. They have to operate on unstable rough terrain, in confined spaces or in sensory-deprived environments filled with smoke or dust. Localization, a common task in mobile robotics which involves determining position and orientation with respect to a given coordinate frame, faces these conditions as well. In this thesis, we describe development of a localization system for tracked mobile robot intended for search and rescue missions. We present a proprioceptive 6-degrees-of-freedom localization system, which arose from the experimental comparison of several possible sensor fusion architectures. The system was modified to incorporate exteroceptive velocity measurements, which significantly improve accuracy by reducing a localization drift. A special attention was given to potential sensor outages and failures, to track slippage that inevitably occurs with this type of robots, to computational demands of the system and to different sampling rates sensory data arrive with. Additionally, we addressed the problem of kinematic models for tracked odometry on rough terrains containing vertical obstacles. Thanks to research projects the robot was designed for, we had access to training facilities used by fire brigades of Italy, Germany and Netherlands. Accuracy and robustness of proposed localization systems was tested in conditions closely resembling those seen in earthquake aftermath and industrial accidents. Datasets used to test our algorithms are publicly available and they are one of the contributions of this thesis. We form this thesis as a compilation of three published papers and one paper in review process

An intelligent multi-floor mobile robot transportation system in life science laboratories

In this dissertation, a new intelligent multi-floor transportation system based on mobile robot is presented to connect the distributed laboratories in multi-floor environment. In the system, new indoor mapping and localization are presented, hybrid path planning is proposed, and an automated doors management system is presented. In addition, a hybrid strategy with innovative floor estimation to handle the elevator operations is implemented. Finally the presented system controls the working processes of the related sub-system. The experiments prove the efficiency of the presented system

Indoor Topological Localization Based on a Novel Deep Learning Technique

Millions of people in the world suffer from vision impairment or vision loss. Traditionally, they rely on guide sticks or dogs to move around and avoid potential obstacles. However, both guide sticks and dogs are passive. They are unable to provide conceptual knowledge or semantic contents of an environment. To address this issue, this paper presents a vision-based cognitive system to support the independence of visually impaired people. More specifically, a 3D indoor semantic map is firstly constructed with a hand-held RGB-D sensor. The constructed map is then deployed for indoor topological localization. Convolutional neural networks are used for both semantic information extraction and location inference. Semantic information is used to further verify localization results and eliminate errors. The topological localization performance can be effectively improved despite significant appearance changes within an environment. Experiments have been conducted to demonstrate that the proposed method can increase both localization accuracy and recall rates. The proposed system can be potentially deployed by visually impaired people to move around safely and have independent life

A software tool for the semi-automatic segmentation of architectural 3D models with semantic annotation and Web fruition

The thorough documentation of Cultural Heritage artifacts is a fundamental concern for management and preservation. In this context, the semantic segmentation and annotation of 3D models of historic buildings is an important modern topic. This work describes a software tool currently under development, for interactive and semi-automatic segmentation, characterization, and annotation of 3D models produced by photogrammetric surveys. The system includes some generic and well-known segmentation approaches, such as region growing and Locally Convex Connected Patches segmentation, but it also contains original code for specific semantic segmentation of parts of buildings, in particular straight stairs and circular-section columns. Furthermore, a method for automatic wall-surface characterization is devoted to rusticated-ashlar detection, in view of masonry-unit segmentation. The software is modular, so allowing easy expandability. It also has tools for data encoding into formats ready for model fruition by Web technologies. These results were partly obtained in collaboration with Corvallis SPA (Padua-Italy, http://www.corvallis.it)

Flexible Supervised Autonomy for Exploration in Subterranean Environments

While the capabilities of autonomous systems have been steadily improving in recent years, these systems still struggle to rapidly explore previously unknown environments without the aid of GPS-assisted navigation. The DARPA Subterranean (SubT) Challenge aimed to fast track the development of autonomous exploration systems by evaluating their performance in real-world underground search-and-rescue scenarios. Subterranean environments present a plethora of challenges for robotic systems, such as limited communications, complex topology, visually-degraded sensing, and harsh terrain. The presented solution enables long-term autonomy with minimal human supervision by combining a powerful and independent single-agent autonomy stack, with higher level mission management operating over a flexible mesh network. The autonomy suite deployed on quadruped and wheeled robots was fully independent, freeing the human supervision to loosely supervise the mission and make high-impact strategic decisions. We also discuss lessons learned from fielding our system at the SubT Final Event, relating to vehicle versatility, system adaptability, and re-configurable communications.Comment: Field Robotics special issue: DARPA Subterranean Challenge, Advancement and Lessons Learned from the Final

Active Vision for Scene Understanding

Visual perception is one of the most important sources of information for both humans and robots. A particular challenge is the acquisition and interpretation of complex unstructured scenes. This work contributes to active vision for humanoid robots. A semantic model of the scene is created, which is extended by successively changing the robot\u27s view in order to explore interaction possibilities of the scene

Emergency Landing Spot Detection for Unmanned Aerial Vehicle

The use and research of Unmanned Aerial Vehicle (UAV) have been increasing over the years due to the applicability in several operations such as search and rescue, delivery, surveillance and others. Considering the increased presence of these vehicles in the airspace, it becomes necessary to reflect on the safety issues or failures that UAV may have and what is the appropriate action to take. Furthermore, in many missions the vehicle will not return to its original location and, in case of fail to achieve the landing spot, need to have onboard capability to estimate the best spot to safely land. The vehicles are susceptible to external disturbance or electromechanical malfunction. In this emergency’s scenarios, UAVs must safely land in a way that will minimize damage to the robot and will not cause any human injury. The suitability of a landing site depends on two main factors: the distance of the aircraft to the landing site and the ground conditions. The ground conditions are all the factors that are relevant when the aircraft is in contact with the ground, such as slope, roughness and presence of obstacles. This dissertation addresses the scenario of finding a safe landing spot during operation. Therefore, the algorithm must be able to classify the incoming data and store the location of suitable areas. Specifically, by processing Light Detection and Ranging (LiDAR) data to identify potential landing zones and evaluating the detected spots continuously given certain conditions. In this dissertation, it was developed a method that analyses geometric features on point cloud data and detects potential good spots. The algorithm uses the Principal Component Analysis (PCA) to find planes in point clouds clusters. The planes that have slope less than a threshold are considered potential landing spots. These spots are then evaluated regarding ground and vehicles conditions such as the distance to the UAV, presence of obstacles, roughness of the area, slope of the spot. The output of the algorithm is the optimum spot to land and can vary during operation.O uso e pesquisa de veículos aéreos não tripulados (VANT) têm aumentado ao longo dos anos devido à aplicabilidade em diversas operações, como busca e salvamento, entrega, vigilância e outras. Considerando a crescente presença desses veículos no espaço aéreo, torna-se necessário refletir sobre os problemas ou falhas de segurança que o veículo pode ter e qual é a ação apropriada a ser tomada. Além disso, em muitas missões, o veículo não retornará ao seu local original e, caso não seja possível alcançar a zona de aterragem, precisa ter a capacidade de estimar o melhor ponto para aterrar em segurança. Os veículos são suscetíveis a perturbações externas ou mau funcionamento eletromecânico. Nesses cenários de emergência, os UAVs precisam aterrar com segurança de forma a minimizar os danos ao robô e não causar ferimentos em pessoas. A adequação de um local de pouso depende de dois fatores principais: a distância do veículo aéreo ao local de pouso e as condições do solo. As condições do solo são todos os fatores relevantes quando a aeronave está em contacto com o solo, como declividade, rugosidade e presença de obstáculos. Esta dissertação aborda o cenário de encontrar um local de pouso seguro durante a operação. Portanto, o algoritmo deve ser capaz de classificar os dados recebidos e armazenar a localização de áreas adequadas. Especificamente, processando dados de LiDAR para identificar possíveis zonas de aterragem e avaliando os pontos detetados continuamente, dadas determinadas condições. Nesta dissertação, foi desenvolvido um método que analisa características geométricas em nuvem de pontos e deteta possíveis bons locais de aterragem. O algoritmo usa a Análise de Componente Principal (PCA) para encontrar planos em clusters de nuvens de pontos. Os planos com inclinação menor que um limite são considerados possíveis pontos de aterragem. Esses pontos são então avaliados quanto às condições do solo e dos veículos, como a distância ao UAV, presença de obstáculos, rugosidade da área, inclinação do ponto. A saída do algoritmo é o local ideal para aterrar e pode variar durante a operação

Computing fast search heuristics for physics-based mobile robot motion planning

Mobile robots are increasingly being employed to assist responders in search and rescue missions. Robots have to navigate in dangerous areas such as collapsed buildings and hazardous sites, which can be inaccessible to humans. Tele-operating the robots can be stressing for the human operators, which are also overloaded with mission tasks and coordination overhead, so it is important to provide the robot with some degree of autonomy, to lighten up the task for the human operator and also to ensure robot safety. Moving robots around requires reasoning, including interpretation of the environment, spatial reasoning, planning of actions (motion), and execution. This is particularly challenging when the environment is unstructured, and the terrain is \textit{harsh}, i.e. not flat and cluttered with obstacles. Approaches reducing the problem to a 2D path planning problem fall short, and many of those who reason about the problem in 3D don't do it in a complete and exhaustive manner. The approach proposed in this thesis is to use rigid body simulation to obtain a more truthful model of the reality, i.e. of the interaction between the robot and the environment. Such a simulation obeys the laws of physics, takes into account the geometry of the environment, the geometry of the robot, and any dynamic constraints that may be in place. The physics-based motion planning approach by itself is also highly intractable due to the computational load required to perform state propagation combined with the exponential blowup of planning; additionally, there are more technical limitations that disallow us to use things such as state sampling or state steering, which are known to be effective in solving the problem in simpler domains. The proposed solution to this problem is to compute heuristics that can bias the search towards the goal, so as to quickly converge towards the solution. With such a model, the search space is a rich space, which can only contain states which are physically reachable by the robot, and also tells us enough information about the safety of the robot itself. The overall result is that by using this framework the robot engineer has a simpler job of encoding the \textit{domain knowledge} which now consists only of providing the robot geometric model plus any constraints