{CurveFusion}: {R}econstructing Thin Structures from {RGBD} Sequences

We introduce CurveFusion, the first approach for high quality scanning of thin structures at interactive rates using a handheld RGBD camera. Thin filament-like structures are mathematically just 1D curves embedded in R^3, and integration-based reconstruction works best when depth sequences (from the thin structure parts) are fused using the object's (unknown) curve skeleton. Thus, using the complementary but noisy color and depth channels, CurveFusion first automatically identifies point samples on potential thin structures and groups them into bundles, each being a group of a fixed number of aligned consecutive frames. Then, the algorithm extracts per-bundle skeleton curves using L1 axes, and aligns and iteratively merges the L1 segments from all the bundles to form the final complete curve skeleton. Thus, unlike previous methods, reconstruction happens via integration along a data-dependent fusion primitive, i.e., the extracted curve skeleton. We extensively evaluate CurveFusion on a range of challenging examples, different scanner and calibration settings, and present high fidelity thin structure reconstructions previously just not possible from raw RGBD sequences

Semi-automated creation of accurate FEM meshes of heritage masonry walls from point cloud data

The structural analysis of buildings requires accurate spatial models. Additionally, pathologies such as settlement-induced damages are paramount in the assessment of heritage assets. This spatial information is used as a basis for Finite Element Method (FEM) meshes to evaluate the stability of the structure. Traditional data acquisition approaches rely on manual measurements which are labor intensive and error prone. Therefore, major simplifications are made to document structures efficiently. The goal of this research is to provide faster and more accurate procedures to capture the spatial information required by a FEM. This paper presents a semi-automated approach to create accurate models of complex heritage buildings for the purpose of structural analysis. By employing non-destructive techniques such as terrestrial laser scanning and photogrammetry, a complex mesh of the structure is created. Also, a methodology is proposed to capture crack information. A stepwise approach is elaborated to illustrate how the spatial information is adapted towards a FEM mesh. The results show a significant difference between the geometry our model and a traditional wire- frame model. Not only does accurate modelling result in deviating loads, it also affects the behavior of the object. Through the proposed approach, experts can develop highly accurate FEM meshes to assess the stability of the structure up to as-built conditionsPostprint (published version

3D Understanding of Deformable Linear Objects: Datasets and Transferability Benchmark

Deformable linear objects are vastly represented in our everyday lives. It is often challenging even for humans to visually understand them, as the same object can be entangled so that it appears completely different. Examples of deformable linear objects include blood vessels and wiring harnesses, vital to the functioning of their corresponding systems, such as the human body and a vehicle. However, no point cloud datasets exist for studying 3D deformable linear objects. Therefore, we are introducing two point cloud datasets, PointWire and PointVessel. We evaluated state-of-the-art methods on the proposed large-scale 3D deformable linear object benchmarks. Finally, we analyzed the generalization capabilities of these methods by conducting transferability experiments on the PointWire and PointVessel datasets

3D model reconstruction using neural gas accelerated on GPU

In this work, we propose the use of the neural gas (NG), a neural network that uses an unsupervised Competitive Hebbian Learning (CHL) rule, to develop a reverse engineering process. This is a simple and accurate method to reconstruct objects from point clouds obtained from multiple overlapping views using low-cost sensors. In contrast to other methods that may need several stages that include downsampling, noise filtering and many other tasks, the NG automatically obtains the 3D model of the scanned objects. To demonstrate the validity of our proposal we tested our method with several models and performed a study of the neural network parameterization computing the quality of representation and also comparing results with other neural methods like growing neural gas and Kohonen maps or classical methods like Voxel Grid. We also reconstructed models acquired by low cost sensors that can be used in virtual and augmented reality environments for redesign or manipulation purposes. Since the NG algorithm has a strong computational cost we propose its acceleration. We have redesigned and implemented the NG learning algorithm to fit it onto Graphics Processing Units using CUDA. A speed-up of 180× faster is obtained compared to the sequential CPU version.This work was partially funded by the Spanish Government DPI2013-40534-R grant

Vid2Curve: Simultaneous Camera Motion Estimation and Thin Structure Reconstruction from an RGB Video

Thin structures, such as wire-frame sculptures, fences, cables, power lines, and tree branches, are common in the real world. It is extremely challenging to acquire their 3D digital models using traditional image-based or depth-based reconstruction methods because thin structures often lack distinct point features and have severe self-occlusion. We propose the first approach that simultaneously estimates camera motion and reconstructs the geometry of complex 3D thin structures in high quality from a color video captured by a handheld camera. Specifically, we present a new curve-based approach to estimate accurate camera poses by establishing correspondences between featureless thin objects in the foreground in consecutive video frames, without requiring visual texture in the background scene to lock on. Enabled by this effective curve-based camera pose estimation strategy, we develop an iterative optimization method with tailored measures on geometry, topology as well as self-occlusion handling for reconstructing 3D thin structures. Extensive validations on a variety of thin structures show that our method achieves accurate camera pose estimation and faithful reconstruction of 3D thin structures with complex shape and topology at a level that has not been attained by other existing reconstruction methods.Comment: Accepted by SIGGRAPH 202

High resolution model mesh and 3D printing of the Gaudí's Porta del Drac

This article intends to explore the limits of scanning with the technology of 3D Laser Scanner and the 3D printing, as an approximation to its application for the survey and the study of singular elements of the architectural heritage. The case study we developed is the Porta del Drac, in the Pavelló Güell, designed by Antoni Gaudí. We divided the process in two parts, one about how to scan and optimize the survey with the Laser Scanner Technology, made with a Faro Forus3D x330 scanner. The second one, about the optimization of the survey as a highresolution mesh to have a scaled 3D model to be printed in 3D, for the musealization of the Verdaguer House of Literature in Vil.la Joana (Barcelona), a project developed by the Museum of History of Barcelona, in tribute to Jacint Verdaguer. In the first place, we propose a methodology for the survey of this atypical model, which is of special interest for several factors: the geometric complexity in relation to the occlusions, the thickness of the metallic surfaces, the hidden internal structure partially seen from the outside, the produced noise in its interior, and the instrumental errors. These factors make the survey process complex from the data collection, having to perform several scans from different positions to cover the entire sculpture, which has a geometry composed of a variety of folds that cause occlusions. Also, the union of the positions and the average of the surfaces is of great relevance, since the elements of the sculpture are constructed by a metal plate of 2mm, therefore, the error in the union of all these many positions must be smaller than this. Moreover, optimization of the cloud has a great difficulty because of the noise created by the instrumental error as it is a metal sculpture and because of noise point clouds that are generated inside the internal folds of the wings, which are made with a welded wire mesh with little spaces between them. Finally, the added difficulty that there is an internal structure between elements of the parts of the Drac that are partially hidden and therefore cannot be recorded. Secondly, we expose the procedures performed to move from a point cloud to an optimal high-resolution mesh to be printed in 3D, adapting it to all the limitations that this printing technique entails. On the one hand, for the meshing process, a previous classification of the point cloud by pieces (wings, chains, mosaics, head ...) is made and an internal structure is re-assembled to avoid floating parts. On the other hand, the selection of the 3D printing technique, in this case FDM (Fused Deposition Modelling), limits the size of the model so it needs to be cut by determined maximum dimension, and also it limits the minimum thickness of the model’s surface, that is to say, the model cannot be directly scaled to the desired size because the 2mm surfaces would be too thin to be printed. This research intends to advance the knowledge of data acquisition, optimization, modelling and 3D printing, with a case study of great complexity. A process that can be systematized and applied to other models.Postprint (published version

Comparative Analysis of Mobile 3D Scanning Technologies for Design, Manufacture of Interior and Exterior Tensile Material Structures and Canvasman Ltd. Case Study

This report aimed to investigate mobile 3D Scanning technologies to improve the 3D data capture and efficiency into Canvasman’s CAD design and manufacturing processes with focus on accurate resolution. The Santander funded Collaborative Venture Fund (CVF) project has provided research, survey data, evaluation and analysis for Canvasman Ltd. on 3D portable scanning hardware and software. The project solutions recommended in this report offers impartial product information on the current appropriate 3D scanning technology that potentially could improve efficiency of data capturing, design and manufacture of interior and exterior spaces, boats, vehicles and other similar constructions for creating and installing flexible coverings and indoor and outdoor structures