Change of Scenery: Unsupervised LiDAR Change Detection for Mobile Robots

This paper presents a fully unsupervised deep change detection approach for mobile robots with 3D LiDAR. In unstructured environments, it is infeasible to define a closed set of semantic classes. Instead, semantic segmentation is reformulated as binary change detection. We develop a neural network, RangeNetCD, that uses an existing point-cloud map and a live LiDAR scan to detect scene changes with respect to the map. Using a novel loss function, existing point-cloud semantic segmentation networks can be trained to perform change detection without any labels or assumptions about local semantics. We demonstrate the performance of this approach on data from challenging terrains; mean intersection over union (mIoU) scores range between 67.4% and 82.2% depending on the amount of environmental structure. This outperforms the geometric baseline used in all experiments. The neural network runs faster than 10Hz and is integrated into a robot's autonomy stack to allow safe navigation around obstacles that intersect the planned path. In addition, a novel method for the rapid automated acquisition of per-point ground-truth labels is described. Covering changed parts of the scene with retroreflective materials and applying a threshold filter to the intensity channel of the LiDAR allows for quantitative evaluation of the change detector.Comment: 7 pages (6 content, 1 references). 7 figures, submitted to the 2024 IEEE International Conference on Robotics and Automation (ICRA

RH-Map: Online Map Construction Framework of Dynamic Objects Removal Based on Region-wise Hash Map Structure

Mobile robots navigating in outdoor environments frequently encounter the issue of undesired traces left by dynamic objects and manifested as obstacles on map, impeding robots from achieving accurate localization and effective navigation. To tackle the problem, a novel map construction framework based on 3D region-wise hash map structure (RH-Map) is proposed, consisting of front-end scan fresher and back-end removal modules, which realizes real-time map construction and online dynamic object removal (DOR). First, a two-layer 3D region-wise hash map structure of map management is proposed for effective online DOR. Then, in scan fresher, region-wise ground plane estimation (R-GPE) is adopted for estimating and preserving ground information and Scan-to-Map Removal (S2M-R) is proposed to discriminate and remove dynamic regions. Moreover, the lightweight back-end removal module maintaining keyframes is proposed for further DOR. As experimentally verified on SemanticKITTI, our proposed framework yields promising performance on online DOR of map construction compared with the state-of-the-art methods. And we also validate the proposed framework in real-world environments

Dynablox: Real-time Detection of Diverse Dynamic Objects in Complex Environments

Real-time detection of moving objects is an essential capability for robots acting autonomously in dynamic environments. We thus propose Dynablox, a novel online mapping-based approach for robust moving object detection in complex environments. The central idea of our approach is to incrementally estimate high confidence free-space areas by modeling and accounting for sensing, state estimation, and mapping limitations during online robot operation. The spatio-temporally conservative free space estimate enables robust detection of moving objects without making any assumptions on the appearance of objects or environments. This allows deployment in complex scenes such as multi-storied buildings or staircases, and for diverse moving objects such as people carrying various items, doors swinging or even balls rolling around. We thoroughly evaluate our approach on real-world data sets, achieving 86% IoU at 17 FPS in typical robotic settings. The method outperforms a recent appearance-based classifier and approaches the performance of offline methods. We demonstrate its generality on a novel data set with rare moving objects in complex environments. We make our efficient implementation and the novel data set available as open-source.Comment: Code released at https://github.com/ethz-asl/dynablo

S2^2MAT: Simultaneous and Self-Reinforced Mapping and Tracking in Dynamic Urban Scenariosorcing Framework for Simultaneous Mapping and Tracking in Unbounded Urban Environments

Despite the increasing prevalence of robots in daily life, their navigation capabilities are still limited to environments with prior knowledge, such as a global map. To fully unlock the potential of robots, it is crucial to enable them to navigate in large-scale unknown and changing unstructured scenarios. This requires the robot to construct an accurate static map in real-time as it explores, while filtering out moving objects to ensure mapping accuracy and, if possible, achieving high-quality pedestrian tracking and collision avoidance. While existing methods can achieve individual goals of spatial mapping or dynamic object detection and tracking, there has been limited research on effectively integrating these two tasks, which are actually coupled and reciprocal. In this work, we propose a solution called S$^2$MAT (Simultaneous and Self-Reinforced Mapping and Tracking) that integrates a front-end dynamic object detection and tracking module with a back-end static mapping module. S$^2$MAT leverages the close and reciprocal interplay between these two modules to efficiently and effectively solve the open problem of simultaneous tracking and mapping in highly dynamic scenarios. We conducted extensive experiments using widely-used datasets and simulations, providing both qualitative and quantitative results to demonstrate S$^2$MAT's state-of-the-art performance in dynamic object detection, tracking, and high-quality static structure mapping. Additionally, we performed long-range robotic navigation in real-world urban scenarios spanning over 7 km, which included challenging obstacles like pedestrians and other traffic agents. The successful navigation provides a comprehensive test of S$^2$MAT's robustness, scalability, efficiency, quality, and its ability to benefit autonomous robots in wild scenarios without pre-built maps.Comment: homepage: https://sites.google.com/view/smat-na

도심도로에서 자율주행차량의 라이다 기반 강건한 위치 및 자세 추정

학위논문(석사) -- 서울대학교대학원 : 공과대학 기계공학부, 2023. 2. 이경수.This paper presents a method for tackling erroneous odometry estimation results from LiDAR-based simultaneous localization and mapping (SLAM) techniques on complex urban roads. Most SLAM techniques estimate sensor odometry through a comparison between measurements from the current and the previous step. As such, a static environment is generally more advantageous for SLAM systems. However, urban environments contain a significant number of dynamic objects, the point clouds of which can noticeably hinder the performance of SLAM systems. As a countermeasure, this paper proposes a 3D LiDAR SLAM system based on static LiDAR point clouds for use in dynamic outdoor urban environments. The proposed method is primarily composed of two parts, moving object detection and pose estimation through 3D LiDAR SLAM. First, moving objects in the vicinity of the ego-vehicle are detected from a referred algorithm based on a geometric model-free approach (GMFA) and a static obstacle map (STOM). GMFA works in conjunction with STOM to estimate the state of moving objects in real-time. The bounding boxes occupied by these moving objects are utilized to remove points corresponding to dynamic objects in the raw LiDAR point clouds. The remaining static points are applied to LiDAR SLAM. The second part of the proposed method describes odometry estimation through referred LiDAR SLAM, LeGO-LOAM. The LeGO-LOAM, a feature-based LiDAR SLAM framework, converts LiDAR point clouds into range images, from which edge and planar points are extracted as features. The range images are further utilized in a preprocessing stage to improve the computation efficiency of the overall algorithm. Additionally, a 6-DOF transformation is utilized, the model equation of which can be obtained by setting a residual to be the distance between an extracted feature of the current step and the corresponding feature geometry of the previous step. The equation is optimized through the Levenberg-Marquardt method. Furthermore, GMFA and LeGO-LOAM operate in parallel to resolve computational delays associated with GMFA. Actual vehicle tests were conducted on urban roads through a test vehicle equipped with a 32-channel 3D LiDAR and a real-time kinematics GPS (RTK GPS). Validations results have shown the proposed method to significantly decrease estimation errors related to moving feature points while securing target output frequency.본 연구는 복잡한 도심 환경에서 라이다 기반 동시적 위치 추정 및 맵핑(Simultaneous localization and mapping, SLAM)의 이동량 추정 오류를 방지하는 방법론을 제안한다. 대부분의 SLAM은 이전 스텝과 현재 스텝의 센서 측정치를 비교하여 자차량의 이동량을 추정한다. 따라서 SLAM에는 정적인 환경이 필수적이다. 그러나 센서는 도심환경에서 동적인 물체에 쉽게 노출되고 동적 물체로부터 출력되는 라이다 점군들은 이동량 추정 성능을 저하시킬 수 있다. 이에, 본 연구는 동적인 도심환경에서 정적인 점군을 기반한 3차원 라이다 SLAM 시스템을 제안하였다. 제안된 방법론은 이동 물체 인지와 3차원 라이다 SLAM을 통한 위치 및 자세 추정으로 구성된다. 우선, 기하학적 모델 프리 접근법과 정지 장애물 맵의 상호 보완적인 관계에 기반한 참고된 알고리즘을 이용해 자차량 주변의 이동 물체의 동적 상태를 실시간으로 추정한다. 그 후, 추정된 이동 물체가 차지하는 경계선을 이용하여 동적 물체에 해당하는 점들을 기존 라이다 점군에서 제거하고, 결과로 얻은 정적인 라이다 점군은 라이다 SLAM에 입력된다. 다음으로, 제안된 방법론은 라이다 SLAM을 통해 자차량의 위치 및 자세를 추정한다. 이를 위해 본 연구는 라이다 SLAM의 프레임워크인 LeGO-LOAM을 채택하였다. 특징점 기반 SLAM인 LeGO-LOAM은 라이다 점군을 거리 기반 이미지로 변환시켜 특징점인 모서리 점과 평면 점을 추출한다. 또한 거리 기반 이미지를 사용한 전처리 과정을 통해 계산 효율을 높인다. 추출된 현재 스텝의 특징점과 이에 대응되는 이전 스텝의 특징점으로 이루어진 기하학적 구조와의 거리를 잔차로 설정하여 6 자유도 변환식에 대한 모델 방정식을 얻을 수 있다. 참고한 LeGO-LOAM은 해당 방정식을 Levenberg-Marquardt 방법을 통해 최적화를 수행한다. 또한, 본 연구는 참고된 인지 모듈의 처리 지연 문제를 보완하기 위해 이동 물체 인지 모듈과 LeGO-LOAM의 병렬 처리 구조를 고안하였다. 실험은 도심환경에서 32채널 3차원 라이다와 고정밀 GPS를 장착한 실험차량으로 진행되었다. 성능 검증 결과, 제안된 방법은 목표 출력 속도를 보장하면서 움직이는 특징점으로 인한 추정 오차를 유의미하게 줄일 수 있었다.Chapter 1. Introduction １ 1.1. Research Motivation １ 1.2. Previous Research ３ 1.2.1. Moving Object Detection ３ 1.2.2. SLAM ４ 1.3. Thesis Objective and Outline １３ Chapter 2. Methodology １５ 2.1. Moving Object Detection & Rejection １５ 2.1.1. Static Obstacle Map １５ 2.1.2. Geometric Model-Free Approach １８ 2.2. LiDAR SLAM ２２ 2.2.1. Segmentation ２２ 2.2.2. Feature Extraction ２３ 2.2.3. LiDAR Odometry and Mapping ２６ 2.2.4. LiDAR SLAM with Static Point Cloud ２８ Chapter 3. Experiments ３０ 3.1. Experimental Setup ３０ 3.2. Error Metrics ３２ 3.3. LiDAR SLAM using Static Point Cloud ３６ Chapter 4. Conclusion ４４ Bibliography ４５석

MotionBEV: Attention-Aware Online LiDAR Moving Object Segmentation with Bird's Eye View based Appearance and Motion Features

Identifying moving objects is an essential capability for autonomous systems, as it provides critical information for pose estimation, navigation, collision avoidance, and static map construction. In this paper, we present MotionBEV, a fast and accurate framework for LiDAR moving object segmentation, which segments moving objects with appearance and motion features in the bird's eye view (BEV) domain. Our approach converts 3D LiDAR scans into a 2D polar BEV representation to improve computational efficiency. Specifically, we learn appearance features with a simplified PointNet and compute motion features through the height differences of consecutive frames of point clouds projected onto vertical columns in the polar BEV coordinate system. We employ a dual-branch network bridged by the Appearance-Motion Co-attention Module (AMCM) to adaptively fuse the spatio-temporal information from appearance and motion features. Our approach achieves state-of-the-art performance on the SemanticKITTI-MOS benchmark. Furthermore, to demonstrate the practical effectiveness of our method, we provide a LiDAR-MOS dataset recorded by a solid-state LiDAR, which features non-repetitive scanning patterns and a small field of view

Design, Deployment, Navigation, and Control of Mobile Robots for Perception and Sensor Data Collection

Aerial robots, including rotary-wing and fixed-wing unmanned aerial vehicles or UAVs, have shown great capabilities in surveying as well as search and rescue from above. However, either rotary-wing or fixed-wing UAVs have nearly insoluble flaws. In order to overcome the under-actuating nature of multi-rotor UAVs, Chapter 2 proposes modeling methods and control schemes for fully-actuated hexacopters. Additionally, rotary-wing robots suffer from limited battery life as well as lack of fail-safe mechanism upon losing motors, while fixed-wing robots lacks the ability to take off and land vertically. Therefore, Chapter 4 proposes a bio-inpired hybrid aerial robot to extend mutli-rotor flight time and fail-safe capability and provide fixed-wing glider with vertical take-off and landing or VTOL capability. Moreover, to extend the flight time and optimize the energy consumption of multi-rotor UAVs, Chapter 3 proposes a multi-disciplinary design optimization based flight trajectory optimizer involving linear rotor inflow models to reduce flight time or energy consumption of specific missions.In terms of unmanned ground vehicles or UGVs used for perception and mapping, there has been a research gap to provide a low-cost, highly agile over-actuated chassis design. Chapter 5 proposes a 3D-printable double Ackermann steering chassis design with 2-wheel standing and balancing capability to fill in this gap. Chapter 6, on the other hand, proposes the system design of a UGV capable of performing perception and mapping in a limited lighting, unstructured, and GPS-denied environment based on a nevertheless nonholonomic chassis, where primary concern becomes the reliability in performing real-time mapping and preservation of solely static environment.The last but not least topic discussed in this dissertation is to promote the role of UAV imagery in earthquake response. In Chapter 7 we combine the traditional UAV plan view perspective with north and east elevation view video data to provide motion estimation in all 6 degrees of freedom, as well as proposing Video Transformer for motion tracking.All in all, with attempts to expand and promote the designs, deployment and control schemes of both aerial and ground mobile robots, this dissertation strives to provide case study results and state-of-the-art methods for future robotics studies