Plane-Based Optimization of Geometry and Texture for RGB-D Reconstruction of Indoor Scenes

We present a novel approach to reconstruct RGB-D indoor scene with plane primitives. Our approach takes as input a RGB-D sequence and a dense coarse mesh reconstructed by some 3D reconstruction method on the sequence, and generate a lightweight, low-polygonal mesh with clear face textures and sharp features without losing geometry details from the original scene. To achieve this, we firstly partition the input mesh with plane primitives, simplify it into a lightweight mesh next, then optimize plane parameters, camera poses and texture colors to maximize the photometric consistency across frames, and finally optimize mesh geometry to maximize consistency between geometry and planes. Compared to existing planar reconstruction methods which only cover large planar regions in the scene, our method builds the entire scene by adaptive planes without losing geometry details and preserves sharp features in the final mesh. We demonstrate the effectiveness of our approach by applying it onto several RGB-D scans and comparing it to other state-of-the-art reconstruction methods.Comment: in International Conference on 3D Vision 2018; Models and Code: see https://github.com/chaowang15/plane-opt-rgbd. arXiv admin note: text overlap with arXiv:1905.0885

PetroSurf3D - A Dataset for high-resolution 3D Surface Segmentation

The development of powerful 3D scanning hardware and reconstruction algorithms has strongly promoted the generation of 3D surface reconstructions in different domains. An area of special interest for such 3D reconstructions is the cultural heritage domain, where surface reconstructions are generated to digitally preserve historical artifacts. While reconstruction quality nowadays is sufficient in many cases, the robust analysis (e.g. segmentation, matching, and classification) of reconstructed 3D data is still an open topic. In this paper, we target the automatic and interactive segmentation of high-resolution 3D surface reconstructions from the archaeological domain. To foster research in this field, we introduce a fully annotated and publicly available large-scale 3D surface dataset including high-resolution meshes, depth maps and point clouds as a novel benchmark dataset to the community. We provide baseline results for our existing random forest-based approach and for the first time investigate segmentation with convolutional neural networks (CNNs) on the data. Results show that both approaches have complementary strengths and weaknesses and that the provided dataset represents a challenge for future research.Comment: CBMI Submission; Dataset and more information can be found at http://lrs.icg.tugraz.at/research/petroglyphsegmentation

Localization in Unstructured Environments: Towards Autonomous Robots in Forests with Delaunay Triangulation

Autonomous harvesting and transportation is a long-term goal of the forest industry. One of the main challenges is the accurate localization of both vehicles and trees in a forest. Forests are unstructured environments where it is difficult to find a group of significant landmarks for current fast feature-based place recognition algorithms. This paper proposes a novel approach where local observations are matched to a general tree map using the Delaunay triangularization as the representation format. Instead of point cloud based matching methods, we utilize a topology-based method. First, tree trunk positions are registered at a prior run done by a forest harvester. Second, the resulting map is Delaunay triangularized. Third, a local submap of the autonomous robot is registered, triangularized and matched using triangular similarity maximization to estimate the position of the robot. We test our method on a dataset accumulated from a forestry site at Lieksa, Finland. A total length of 2100\,m of harvester path was recorded by an industrial harvester with a 3D laser scanner and a geolocation unit fixed to the frame. Our experiments show a 12\,cm s.t.d. in the location accuracy and with real-time data processing for speeds not exceeding 0.5\,m/s. The accuracy and speed limit is realistic during forest operations

optimización da planificación de adquisición de datos LIDAR cara ó modelado 3D de interiores

The main objective of this doctoral thesis is the design, validation and implementation of methodologies that allow the geometric and topological modelling of navigable spaces, whether inside buildings or urban environments, to be integrated into three-dimensional geographic information systems (GIS-3D). The input data of this work will consist mainly of point clouds (which can be classified) acquired by LiDAR systems both indoors and outdoors. In addition, the use of BIM infrastructure models and cadastral maps is proposed depending on their availability. Point clouds provide a large amount of environmental information with high accuracy compared to data offered by other acquisition technologies. However, the lack of data structure and volume requires a great deal of processing effort. For this reason, the first step is to structure the data by dividing the input cloud into simpler entities that facilitate subsequent processes. For this first division, the physical elements present in the cloud will be considered, since they can be walls in the case of interior environments or kerbs in the case of exteriors. In order to generate navigation routes adapted to different mobile agents, the next objective will try to establish a semantic subdivision of space according to the functionalities of space. In the case of internal environments, it is possible to use BIM models to evaluate the results and the use of cadastral maps that support the division of the urban environment. Once the navigable space is divided, the design of topologically coherent navigation networks will be parameterized both geometrically and topologically. For this purpose, several spatial discretization techniques, such as 3D tessellations, will be studied to facilitate the establishment of topological relationships, adjacency, connectivity and inclusion between subspaces. Based on the geometric characterization and the topological relations established in the previous phase, the creation of three-dimensional navigation networks with multimodal support will be addressed and different levels of detail will be considered according to the mobility specifications of each agent and its purpose. Finally, the possibility of integrating the networks generated in a GIS-3D visualization system will be considered. For the correct visualization, the level of detail can be adjusted according to geometry and semantics. Aspects such as the type of user or transport, mobility, rights of access to spaces, etc. They must be considered at all times.El objetivo principal de esta tesis doctoral es el diseño, la validación y la implementación de metodologías que permitan el modelado geométrico y topológico de espacios navegables, ya sea de interiores de edificios o entornos urbanos, para integrarse en sistemas de información geográfica tridimensional (SIG). -3D). Los datos de partida de este trabajo consistirán principalmente en nubes de puntos (que pueden estar clasificados) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Además, se propone el uso de modelos BIM de infraestructuras y mapas catastrales en función de su disponibilidad. Las nubes de puntos proporcionan una gran cantidad de información del entorno con gran precisión con respecto a los datos ofrecidos por otras tecnologías de adquisición. Sin embargo, la falta de estructura de datos y su volumen requiere un gran esfuerzo de procesamiento. Por este motivo, el primer paso que se debe realizar consiste en estructurar los datos dividiendo la nube de entrada en entidades más simples que facilitan los procesos posteriores. Para esta primera división se considerarán los elementos físicos presentes en la nube, ya que pueden ser paredes en el caso de entornos interiores o bordillos en el caso de los exteriores. Con el propósito de generar rutas de navegación adaptadas a diferentes agentes móviles, el próximo objetivo intentará establecer una subdivisión semántica del espacio de acuerdo con las funcionalidades del espacio. En el caso de entornos internos, es posible utilizar modelos BIM para evaluar los resultados y el uso de mapas catastrales que sirven de apoyo en la división del entorno urbano. Una vez que se divide el espacio navegable, se parametrizará tanto geométrica como topológicamente al diseño de redes de navegación topológicamente coherentes. Para este propósito, se estudiarán varias técnicas de discretización espacial, como las teselaciones 3D, para facilitar el establecimiento de relaciones topológicas, la adyacencia, la conectividad y la inclusión entre subespacios. A partir de la caracterización geométrica y las relaciones topológicas establecidas en la fase anterior, se abordará la creación de redes de navegación tridimensionales con soporte multimodal y se considerarán diversos niveles de detalle según las especificaciones de movilidad de cada agente y su propósito. Finalmente, se contemplará la posibilidad de integrar las redes generadas en un sistema de visualización tridimensional 3D SIG 3D. Para la correcta visualización, el nivel de detalle se puede ajustar en función de la geometría y la semántica. Aspectos como el tipo de usuario o transporte, movilidad, derechos de acceso a espacios, etc. Deben ser considerados en todo momento.O obxectivo principal desta tese doutoral é o deseño, validación e implementación de metodoloxías que permitan o modelado xeométrico e topolóxico de espazos navegables, ben sexa de interiores de edificios ou de entornos urbanos, ca fin de seren integrados en Sistemas de Información Xeográfica tridimensionais (SIX-3D). Os datos de partida deste traballo constarán principalmente de nubes de puntos (que poden estar clasificadas) adquiridas por sistemas LiDAR tanto en interiores como en exteriores. Ademáis plantease o uso de modelos BIM de infraestruturas e mapas catastrais dependendo da súa dispoñibilidade. As nubes de puntos proporcionan unha gran cantidade de información do entorno cunha gran precisión respecto os datos que ofrecen outras tecnoloxías de adquisición. Sen embargo, a falta de estrutura dos datos e a seu volume esixe un amplo esforzo de procesado. Por este motivo o primeiro paso a levar a cabo consiste nunha estruturación dos datos mediante a división da nube de entrada en entidades máis sinxelas que faciliten os procesos posteriores. Para esta primeira división consideraranse elementos físicos presentes na nube como poden ser paredes no caso de entornos interiores ou bordillos no caso de exteriores. Coa finalidade de xerar rutas de navegación adaptadas a distintos axentes móbiles, o seguinte obxectivo tratará de establecer unha subdivisión semántica do espazo de acordo as funcionalidades do espazo. No caso de entornos interiores plantease a posibilidade de empregar modelos BIM para avaliar os resultados e o uso de mapas catastrais que sirvan de apoio na división do entorno urbano. Unha vez divido o espazo navigable parametrizarase tanto xeométricamente como topolóxicamene de cara ao deseño de redes de navegación topolóxicamente coherentes. Para este fin estudaranse varias técnicas de discretización de espazos como como son as teselacións 3D co obxectivo de facilitar establecer relacións topolóxicas, de adxacencia, conectividade e inclusión entre subespazos. A partir da caracterización xeométrica e das relación topolóxicas establecidas na fase previa abordarase a creación de redes de navegación tridimensionais con soporte multi-modal e considerando varios niveis de detalle de acordo as especificacións de mobilidade de cada axente e a súa finalidade. Finalmente comtemplarase a posibilidade de integrar as redes xeradas nun sistema SIX 3D visualización tridimensional. Para a correcta visualización o nivel de detalle poderá axustarse en base a xeometría e a semántica. Aspectos como o tipo de usuario ou transporte, mobilidade, dereitos de acceso a espazos, etc. deberán ser considerados en todo momento

Enabling Multi-LiDAR Sensing in GNSS-Denied Environments: SLAM Dataset, Benchmark, and UAV Tracking with LiDAR-as-a-camera

The rise of Light Detection and Ranging (LiDAR) sensors has profoundly impacted industries ranging from automotive to urban planning. As these sensors become increasingly affordable and compact, their applications are diversifying, driving precision, and innovation. This thesis delves into LiDAR's advancements in autonomous robotic systems, with a focus on its role in simultaneous localization and mapping (SLAM) methodologies and LiDAR as a camera-based tracking for Unmanned Aerial Vehicles (UAV). Our contributions span two primary domains: the Multi-Modal LiDAR SLAM Benchmark, and the LiDAR-as-a-camera UAV Tracking. In the former, we have expanded our previous multi-modal LiDAR dataset by adding more data sequences from various scenarios. In contrast to the previous dataset, we employ different ground truth-generating approaches. We propose a new multi-modal multi-lidar SLAM-assisted and ICP-based sensor fusion method for generating ground truth maps. Additionally, we also supplement our data with new open road sequences with GNSS-RTK. This enriched dataset, supported by high-resolution LiDAR, provides detailed insights through an evaluation of ten configurations, pairing diverse LiDAR sensors with state-of-the-art SLAM algorithms. In the latter contribution, we leverage a custom YOLOv5 model trained on panoramic low-resolution images from LiDAR reflectivity (LiDAR-as-a-camera) to detect UAVs, demonstrating the superiority of this approach over point cloud or image-only methods. Additionally, we evaluated the real-time performance of our approach on the Nvidia Jetson Nano, a popular mobile computing platform. Overall, our research underscores the transformative potential of integrating advanced LiDAR sensors with autonomous robotics. By bridging the gaps between different technological approaches, we pave the way for more versatile and efficient applications in the future

Comparison of low-cost handheld LiDAR-based SLAM systems for mapping underground tunnels

The use of mobile mapping technologies (MMT) has become increasingly popular across various applications such as forestry, cultural heritage, mining, and civil engineering. While Simultaneous Localization and Mapping (SLAM) algorithms have greatly improved in recent years with regards to accuracy, robustness, and cooperativity, it is important to understand the limitations and strengths of each metrological measurement method to ensure the provision of 3D data of appropriate quality for the selected application. In this study, we perform a comparative analysis of three LiDAR-based handheld mobile mapping systems with survey-grade reference point clouds in a challenging test area of a partially collapsed underground tunnel. We investigate various aspects of 3D data quality, including accuracy and completeness, and present an improved method for 3D data completeness assessment aimed at evaluating SLAM-derived point clouds. The results demonstrate unique and diverse strengths and shortcomings of the tested mapping systems, which provide valuable guidelines for selecting an appropriate system for subterranean applications