Self-correction of 3D reconstruction from multi-view stereo images

We present a self-correction approach to improving the 3D reconstruction of a multi-view 3D photogrammetry system. The self-correction approach has been able to repair the reconstructed 3D surface damaged by depth discontinuities. Due to self-occlusion, multi-view range images have to be acquired and integrated into a watertight nonredundant mesh model in order to cover the extended surface of an imaged object. The integrated surface often suffers from “dent” artifacts produced by depth discontinuities in the multi-view range images. In this paper we propose a novel approach to correcting the 3D integrated surface such that the dent artifacts can be repaired automatically. We show examples of 3D reconstruction to demonstrate the improvement that can be achieved by the self-correction approach. This self-correction approach can be extended to integrate range images obtained from alternative range capture devices

Novel methods for real-time 3D facial recognition

In this paper we discuss our approach to real-time 3D face recognition. We argue the need for real time operation in a realistic scenario and highlight the required pre- and post-processing operations for effective 3D facial recognition. We focus attention to some operations including face and eye detection, and fast post-processing operations such as hole filling, mesh smoothing and noise removal. We consider strategies for hole filling such as bilinear and polynomial interpolation and Laplace and conclude that bilinear interpolation is preferred. Gaussian and moving average smoothing strategies are compared and it is shown that moving average can have the edge over Gaussian smoothing. The regions around the eyes normally carry a considerable amount of noise and strategies for replacing the eyeball with a spherical surface and the use of an elliptical mask in conjunction with hole filling are compared. Results show that the elliptical mask with hole filling works well on face models and it is simpler to implement. Finally performance issues are considered and the system has demonstrated to be able to perform real-time 3D face recognition in just over 1s 200ms per face model for a small database

Scalable Surface Reconstruction from Point Clouds with Extreme Scale and Density Diversity

In this paper we present a scalable approach for robustly computing a 3D surface mesh from multi-scale multi-view stereo point clouds that can handle extreme jumps of point density (in our experiments three orders of magnitude). The backbone of our approach is a combination of octree data partitioning, local Delaunay tetrahedralization and graph cut optimization. Graph cut optimization is used twice, once to extract surface hypotheses from local Delaunay tetrahedralizations and once to merge overlapping surface hypotheses even when the local tetrahedralizations do not share the same topology.This formulation allows us to obtain a constant memory consumption per sub-problem while at the same time retaining the density independent interpolation properties of the Delaunay-based optimization. On multiple public datasets, we demonstrate that our approach is highly competitive with the state-of-the-art in terms of accuracy, completeness and outlier resilience. Further, we demonstrate the multi-scale potential of our approach by processing a newly recorded dataset with 2 billion points and a point density variation of more than four orders of magnitude - requiring less than 9GB of RAM per process.Comment: This paper was accepted to the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. The copyright was transfered to IEEE (ieee.org). The official version of the paper will be made available on IEEE Xplore (R) (ieeexplore.ieee.org). This version of the paper also contains the supplementary material, which will not appear IEEE Xplore (R

Towards recovery of complex shapes in meshes using digital images for reverse engineering applications

When an object owns complex shapes, or when its outer surfaces are simply inaccessible, some of its parts may not be captured during its reverse engineering. These deficiencies in the point cloud result in a set of holes in the reconstructed mesh. This paper deals with the use of information extracted from digital images to recover missing areas of a physical object. The proposed algorithm fills in these holes by solving an optimization problem that combines two kinds of information: (1) the geometric information available on the surrounding of the holes, (2) the information contained in an image of the real object. The constraints come from the image irradiance equation, a first-order non-linear partial differential equation that links the position of the mesh vertices to the light intensity of the image pixels. The blending conditions are satisfied by using an objective function based on a mechanical model of bar network that simulates the curvature evolution over the mesh. The inherent shortcomings both to the current holefilling algorithms and the resolution of the image irradiance equations are overcom

Stereoscopic Polar Plume Reconstructions from Stereo/Secchi Images

We present stereoscopic reconstructions of the location and inclination of polar plumes of two data sets based on the two simultaneously recorded images taken by the EUVI telescopes in the SECCHI instrument package onboard the \emph{STEREO (Solar TErrestrial RElations Observatory)} spacecraft. The ten plumes investigated show a superradial expansion in the coronal hole in 3D which is consistent with the 2D results. Their deviations from the local meridian planes are rather small with an average of $6.47^{\circ}$. By comparing the reconstructed plumes with a dipole field with its axis along the solar rotation axis, it is found that plumes are inclined more horizontally than the dipole field. The lower the latitude is, the larger is the deviation from the dipole field. The relationship between plumes and bright points has been investigated and they are not always associated. For the first data set, based on the 3D height of plumes and the electron density derived from SUMER/\emph{SOHO} Si {\sc viii} line pair, we found that electron densities along the plumes decrease with height above the solar surface. The temperature obtained from the density scale height is 1.6 to 1.8 times larger than the temperature obtained from Mg {\sc ix} line ratios. We attribute this discrepancy to a deviation of the electron and the ion temperatures. Finally, we have found that the outflow speeds studied in the O {\sc vi} line in the plumes corrected by the angle between the line of sight and the plume orientation are quite small with a maximum of 10 $\mathrm{km s^{-1}}$. It is unlikely that plumes are a dominant contributor to the fast solar wind.Comment: 25 pages, 13 figure