Self-correction of 3D reconstruction from multi-view stereo images

We present a self-correction approach to improving the 3D reconstruction of a multi-view 3D photogrammetry system. The self-correction approach has been able to repair the reconstructed 3D surface damaged by depth discontinuities. Due to self-occlusion, multi-view range images have to be acquired and integrated into a watertight nonredundant mesh model in order to cover the extended surface of an imaged object. The integrated surface often suffers from “dent” artifacts produced by depth discontinuities in the multi-view range images. In this paper we propose a novel approach to correcting the 3D integrated surface such that the dent artifacts can be repaired automatically. We show examples of 3D reconstruction to demonstrate the improvement that can be achieved by the self-correction approach. This self-correction approach can be extended to integrate range images obtained from alternative range capture devices

3D Shape Reconstruction from Multiple Range Image Views

Shape reconstruction of different three dimensional objects using multiple range images has evolved recently within the recent past. In this research shape reconstruction of a three dimensional object using multiple range image views is investigated. Range images were captured using the Waikato Range Imager. This range images camera is novel in that it uses heterodyne imaging and is capable of acquiring range images with precision less than a millimeter simultaneously over a full field. Multiple views of small objects were taken and the FastRBF was explored as a mean of registration and surface rendering. For comparison to the real range data, simulated range data under noise free condition were also generated and reconstructed with the FastRBF tool box. The registration and reconstruction of simple object was performed using different views with the FastRBF toolbox. Analysis of the registration process showed that the translation error produced due to distortion during registration of different views hinders the process of reconstructing a complete surface. While analyzing the shape reconstruction using the FastRBF tool it is also determined that a small change in accuracy values can affect the interpolation drastically. Results of reconstruction of a real 3D object from multiple views are shown

Atomic-scale structure of the SrTiO3(001)-c(6x2) reconstruction: Experiments and first-principles calculations

The c(6x2) is a reconstruction of the SrTiO3(001) surface that is formed between 1050-1100oC in oxidizing annealing conditions. This work proposes a model for the atomic structure for the c(6x2) obtained through a combination of results from transmission electron diffraction, surface x-ray diffraction, direct methods analysis, computational combinational screening, and density functional theory. As it is formed at high temperatures, the surface is complex and can be described as a short-range ordered phase featuring microscopic domains composed of four main structural motifs. Additionally, non-periodic TiO2 units are present on the surface. Simulated scanning tunneling microscopy images based on the electronic structure calculations are consistent with experimental images

Development and Application of 3-Dimensional Transmission Electron Microscopy (3D-TEM) for the Characterization of Metal-Zeolite Catalyst Systems

With electron tomography (3D-TEM) a 3D-reconstruction is calculated from a series of TEM images taken at a tilt angle range (tilting range) of +70° to -70°. The reconstruction can be visualized with contour surfaces that give information about the surface of the sample as well as with slices through the reconstruction that give detailed information on the interior of the sample. Electron tomography gives much more information than Scanning Electron Microscopy (SEM), since SEM gives only information about the surface of a sample. As a case study, the imaging of silver clusters on zeolite NaY is given. The reconstruction shows silver particles at the external surface as well as a silver particle in a mesopore of the zeolite crystallite. It is concluded that 3D-TEM comprises a breakthrough in the characterization of nano-structured solid catalysts

Reconstructing Surfaces by Volumetric Regularization Using Radial Basis Functions

We present a new method of surface reconstruction that generates smooth and seamless models from sparse, noisy, nonuniform, and low resolution range data. Data acquisition techniques from computer vision, such as stereo range images and space carving, produce 3D point sets that are imprecise and nonuniform when compared to laser or optical range scanners. Traditional reconstruction algorithms designed for dense and precise data do not produce smooth reconstructions when applied to vision-based data sets. Our method constructs a 3D implicit surface, formulated as a sum of weighted radial basis functions. We achieve three primary advantages over existing algorithms: (1) the implicit functions we construct estimate the surface well in regions where there is little data, (2) the reconstructed surface is insensitive to noise in data acquisition because we can allow the surface to approximate, rather than exactly interpolate, the data, and (3) the reconstructed surface is locally detailed, yet globally smooth, because we use radial basis functions that achieve multiple orders of smoothness

Characterization and Improvement of the Image Quality of the Data Taken with the Infrared Camera (IRC) Mid-Infrared Channels onboard AKARI

Mid-infrared images frequently suffer artifacts and extended point spread functions (PSFs). We investigate the characteristics of the artifacts and the PSFs in images obtained with the Infrared Camera (IRC) onboard AKARI at four mid-infrared bands of the S7 (7{\mu}m), S11 (11{\mu}m), L15 (15{\mu}m), and L24 (24 {\mu}m). Removal of the artifacts significantly improves the reliability of the ref- erence data for flat-fielding at the L15 and L24 bands. A set of models of the IRC PSFs is also constructed from on-orbit data. These PSFs have extended components that come from diffraction and scattering within the detector arrays. We estimate the aperture correction factors for point sources and the surface brightness correction factors for diffuse sources. We conclude that the surface brightness correction factors range from 0.95 to 0.8, taking account of the extended component of the PSFs. To correct for the extended PSF effects for the study of faint structures, we also develop an image reconstruction method, which consists of the deconvolution with the PSF and the convolution with an appropriate Gaussian. The appropriate removal of the artifacts, improved flat-fielding, and image reconstruction with the extended PSFs enable us to investigate de- tailed structures of extended sources in IRC mid-infrared images.Comment: 35 pages, 15 figures, accepted for publication in PAS

Manifold Constrained Low-Rank Decomposition

Low-rank decomposition (LRD) is a state-of-the-art method for visual data reconstruction and modelling. However, it is a very challenging problem when the image data contains significant occlusion, noise, illumination variation, and misalignment from rotation or viewpoint changes. We leverage the specific structure of data in order to improve the performance of LRD when the data are not ideal. To this end, we propose a new framework that embeds manifold priors into LRD. To implement the framework, we design an alternating direction method of multipliers (ADMM) method which efficiently integrates the manifold constraints during the optimization process. The proposed approach is successfully used to calculate low-rank models from face images, hand-written digits and planar surface images. The results show a consistent increase of performance when compared to the state-of-the-art over a wide range of realistic image misalignments and corruptions

Surface Modeling and Analysis Using Range Images: Smoothing, Registration, Integration, and Segmentation

This dissertation presents a framework for 3D reconstruction and scene analysis, using a set of range images. The motivation for developing this framework came from the needs to reconstruct the surfaces of small mechanical parts in reverse engineering tasks, build a virtual environment of indoor and outdoor scenes, and understand 3D images. The input of the framework is a set of range images of an object or a scene captured by range scanners. The output is a triangulated surface that can be segmented into meaningful parts. A textured surface can be reconstructed if color images are provided. The framework consists of surface smoothing, registration, integration, and segmentation. Surface smoothing eliminates the noise present in raw measurements from range scanners. This research proposes area-decreasing flow that is theoretically identical to the mean curvature flow. Using area-decreasing flow, there is no need to estimate the curvature value and an optimal step size of the flow can be obtained. Crease edges and sharp corners are preserved by an adaptive scheme. Surface registration aligns measurements from different viewpoints in a common coordinate system. This research proposes a new surface representation scheme named point fingerprint. Surfaces are registered by finding corresponding point pairs in an overlapping region based on fingerprint comparison. Surface integration merges registered surface patches into a whole surface. This research employs an implicit surface-based integration technique. The proposed algorithm can generate watertight models by space carving or filling the holes based on volumetric interpolation. Textures from different views are integrated inside a volumetric grid. Surface segmentation is useful to decompose CAD models in reverse engineering tasks and help object recognition in a 3D scene. This research proposes a watershed-based surface mesh segmentation approach. The new algorithm accurately segments the plateaus by geodesic erosion using fast marching method. The performance of the framework is presented using both synthetic and real world data from different range scanners. The dissertation concludes by summarizing the development of the framework and then suggests future research topics

Elevation Estimation-Driven Building 3D Reconstruction from Single-View Remote Sensing Imagery

Building 3D reconstruction from remote sensing images has a wide range of applications in smart cities, photogrammetry and other fields. Methods for automatic 3D urban building modeling typically employ multi-view images as input to algorithms to recover point clouds and 3D models of buildings. However, such models rely heavily on multi-view images of buildings, which are time-intensive and limit the applicability and practicality of the models. To solve these issues, we focus on designing an efficient DSM estimation-driven reconstruction framework (Building3D), which aims to reconstruct 3D building models from the input single-view remote sensing image. First, we propose a Semantic Flow Field-guided DSM Estimation (SFFDE) network, which utilizes the proposed concept of elevation semantic flow to achieve the registration of local and global features. Specifically, in order to make the network semantics globally aware, we propose an Elevation Semantic Globalization (ESG) module to realize the semantic globalization of instances. Further, in order to alleviate the semantic span of global features and original local features, we propose a Local-to-Global Elevation Semantic Registration (L2G-ESR) module based on elevation semantic flow. Our Building3D is rooted in the SFFDE network for building elevation prediction, synchronized with a building extraction network for building masks, and then sequentially performs point cloud reconstruction, surface reconstruction (or CityGML model reconstruction). On this basis, our Building3D can optionally generate CityGML models or surface mesh models of the buildings. Extensive experiments on ISPRS Vaihingen and DFC2019 datasets on the DSM estimation task show that our SFFDE significantly improves upon state-of-the-arts. Furthermore, our Building3D achieves impressive results in the 3D point cloud and 3D model reconstruction process