NICP: Dense normal based point cloud registration

In this paper we present a novel on-line method to recursively align point clouds. By considering each point together with the local features of the surface (normal and curvature), our method takes advantage of the 3D structure around the points for the determination of the data association between two clouds. The algorithm relies on a least squares formulation of the alignment problem, that minimizes an error metric depending on these surface characteristics. We named the approach Normal Iterative Closest Point (NICP in short). Extensive experiments on publicly available benchmark data show that NICP outperforms other state-of-the-art approaches

Data Fusion of Objects Using Techniques Such as Laser Scanning, Structured Light and Photogrammetry for Cultural Heritage Applications

In this paper we present a semi-automatic 2D-3D local registration pipeline capable of coloring 3D models obtained from 3D scanners by using uncalibrated images. The proposed pipeline exploits the Structure from Motion (SfM) technique in order to reconstruct a sparse representation of the 3D object and obtain the camera parameters from image feature matches. We then coarsely register the reconstructed 3D model to the scanned one through the Scale Iterative Closest Point (SICP) algorithm. SICP provides the global scale, rotation and translation parameters, using minimal manual user intervention. In the final processing stage, a local registration refinement algorithm optimizes the color projection of the aligned photos on the 3D object removing the blurring/ghosting artefacts introduced due to small inaccuracies during the registration. The proposed pipeline is capable of handling real world cases with a range of characteristics from objects with low level geometric features to complex ones

Real-time High Resolution Fusion of Depth Maps on GPU

A system for live high quality surface reconstruction using a single moving depth camera on a commodity hardware is presented. High accuracy and real-time frame rate is achieved by utilizing graphics hardware computing capabilities via OpenCL and by using sparse data structure for volumetric surface representation. Depth sensor pose is estimated by combining serial texture registration algorithm with iterative closest points algorithm (ICP) aligning obtained depth map to the estimated scene model. Aligned surface is then fused into the scene. Kalman filter is used to improve fusion quality. Truncated signed distance function (TSDF) stored as block-based sparse buffer is used to represent surface. Use of sparse data structure greatly increases accuracy of scanned surfaces and maximum scanning area. Traditional GPU implementation of volumetric rendering and fusion algorithms were modified to exploit sparsity to achieve desired performance. Incorporation of texture registration for sensor pose estimation and Kalman filter for measurement integration improved accuracy and robustness of scanning process

Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

Structured Light-Based 3D Reconstruction System for Plants.

Camera-based 3D reconstruction of physical objects is one of the most popular computer vision trends in recent years. Many systems have been built to model different real-world subjects, but there is lack of a completely robust system for plants. This paper presents a full 3D reconstruction system that incorporates both hardware structures (including the proposed structured light system to enhance textures on object surfaces) and software algorithms (including the proposed 3D point cloud registration and plant feature measurement). This paper demonstrates the ability to produce 3D models of whole plants created from multiple pairs of stereo images taken at different viewing angles, without the need to destructively cut away any parts of a plant. The ability to accurately predict phenotyping features, such as the number of leaves, plant height, leaf size and internode distances, is also demonstrated. Experimental results show that, for plants having a range of leaf sizes and a distance between leaves appropriate for the hardware design, the algorithms successfully predict phenotyping features in the target crops, with a recall of 0.97 and a precision of 0.89 for leaf detection and less than a 13-mm error for plant size, leaf size and internode distance

Suitability of ground-based SfM-MVS for monitoring glacial and periglacial processes

Photo-based surface reconstruction is rapidly emerging as an alternative survey technique to lidar (light detection and ranging) in many fields of geoscience fostered by the recent development of computer vision algorithms such as structure from motion (SfM) and dense image matching such as multi-view stereo (MVS). The objectives of this work are to test the suitability of the ground-based SfM-MVS approach for calculating the geodetic mass balance of a 2.1km2 glacier and for detecting the surface displacement of a neighbouring active rock glacier located in the eastern Italian Alps. The photos were acquired in 2013 and 2014 using a digital consumer-grade camera during single-day field surveys. Airborne laser scanning (ALS, otherwise known as airborne lidar) data were used as benchmarks to estimate the accuracy of the photogrammetric digital elevation models (DEMs) and the reliability of the method. The SfM-MVS approach enabled the reconstruction of high-quality DEMs, which provided estimates of glacial and periglacial processes similar to those achievable using ALS. In stable bedrock areas outside the glacier, the mean and the standard deviation of the elevation difference between the SfM-MVS DEM and the ALS DEM was-0.42 \ub1 1.72 and 0.03 \ub1 0.74 m in 2013 and 2014, respectively. The overall pattern of elevation loss and gain on the glacier were similar with both methods, ranging between-5.53 and + 3.48 m. In the rock glacier area, the elevation difference between the SfM-MVS DEM and the ALS DEM was 0.02 \ub1 0.17 m. The SfM-MVS was able to reproduce the patterns and the magnitudes of displacement of the rock glacier observed by the ALS, ranging between 0.00 and 0.48 m per year. The use of natural targets as ground control points, the occurrence of shadowed and low-contrast areas, and in particular the suboptimal camera network geometry imposed by the morphology of the study area were the main factors affecting the accuracy of photogrammetric DEMs negatively. Technical improvements such as using an aerial platform and/or placing artificial targets could significantly improve the results but run the risk of being more demanding in terms of costs and logistics