228,362 research outputs found
HR-NeuS: Recovering High-Frequency Surface Geometry via Neural Implicit Surfaces
Recent advances in neural implicit surfaces for multi-view 3D reconstruction
primarily focus on improving large-scale surface reconstruction accuracy, but
often produce over-smoothed geometries that lack fine surface details. To
address this, we present High-Resolution NeuS (HR-NeuS), a novel neural
implicit surface reconstruction method that recovers high-frequency surface
geometry while maintaining large-scale reconstruction accuracy. We achieve this
by utilizing (i) multi-resolution hash grid encoding rather than positional
encoding at high frequencies, which boosts our model's expressiveness of local
geometry details; (ii) a coarse-to-fine algorithmic framework that selectively
applies surface regularization to coarse geometry without smoothing away fine
details; (iii) a coarse-to-fine grid annealing strategy to train the network.
We demonstrate through experiments on DTU and BlendedMVS datasets that our
approach produces 3D geometries that are qualitatively more detailed and
quantitatively of similar accuracy compared to previous approaches
Building with Drones: Accurate 3D Facade Reconstruction using MAVs
Automatic reconstruction of 3D models from images using multi-view
Structure-from-Motion methods has been one of the most fruitful outcomes of
computer vision. These advances combined with the growing popularity of Micro
Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools
ubiquitous for large number of Architecture, Engineering and Construction
applications among audiences, mostly unskilled in computer vision. However, to
obtain high-resolution and accurate reconstructions from a large-scale object
using SfM, there are many critical constraints on the quality of image data,
which often become sources of inaccuracy as the current 3D reconstruction
pipelines do not facilitate the users to determine the fidelity of input data
during the image acquisition. In this paper, we present and advocate a
closed-loop interactive approach that performs incremental reconstruction in
real-time and gives users an online feedback about the quality parameters like
Ground Sampling Distance (GSD), image redundancy, etc on a surface mesh. We
also propose a novel multi-scale camera network design to prevent scene drift
caused by incremental map building, and release the first multi-scale image
sequence dataset as a benchmark. Further, we evaluate our system on real
outdoor scenes, and show that our interactive pipeline combined with a
multi-scale camera network approach provides compelling accuracy in multi-view
reconstruction tasks when compared against the state-of-the-art methods.Comment: 8 Pages, 2015 IEEE International Conference on Robotics and
Automation (ICRA '15), Seattle, WA, US
NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction
Recent methods for neural surface representation and rendering, for example
NeuS, have demonstrated remarkably high-quality reconstruction of static
scenes. However, the training of NeuS takes an extremely long time (8 hours),
which makes it almost impossible to apply them to dynamic scenes with thousands
of frames. We propose a fast neural surface reconstruction approach, called
NeuS2, which achieves two orders of magnitude improvement in terms of
acceleration without compromising reconstruction quality. To accelerate the
training process, we integrate multi-resolution hash encodings into a neural
surface representation and implement our whole algorithm in CUDA. We also
present a lightweight calculation of second-order derivatives tailored to our
networks (i.e., ReLU-based MLPs), which achieves a factor two speed up. To
further stabilize training, a progressive learning strategy is proposed to
optimize multi-resolution hash encodings from coarse to fine. In addition, we
extend our method for reconstructing dynamic scenes with an incremental
training strategy. Our experiments on various datasets demonstrate that NeuS2
significantly outperforms the state-of-the-arts in both surface reconstruction
accuracy and training speed. The video is available at
https://vcai.mpi-inf.mpg.de/projects/NeuS2/
3D Shape Reconstruction from Sketches via Multi-view Convolutional Networks
We propose a method for reconstructing 3D shapes from 2D sketches in the form
of line drawings. Our method takes as input a single sketch, or multiple
sketches, and outputs a dense point cloud representing a 3D reconstruction of
the input sketch(es). The point cloud is then converted into a polygon mesh. At
the heart of our method lies a deep, encoder-decoder network. The encoder
converts the sketch into a compact representation encoding shape information.
The decoder converts this representation into depth and normal maps capturing
the underlying surface from several output viewpoints. The multi-view maps are
then consolidated into a 3D point cloud by solving an optimization problem that
fuses depth and normals across all viewpoints. Based on our experiments,
compared to other methods, such as volumetric networks, our architecture offers
several advantages, including more faithful reconstruction, higher output
surface resolution, better preservation of topology and shape structure.Comment: 3DV 2017 (oral
An integrated approach for reconstructing a surface model of the proximal femur from sparse input data and a multi-resolution point distribution model: an in vitro study
Background: Accurate reconstruction of a patient-specific surface model of the proximal femur from preoperatively or intraoperatively available sparse data plays an important role in planning and supporting various computer-assisted surgical procedures. Methods: In this paper, we present an integrated approach using a multi-resolution point distribution model (MR-PDM) to reconstruct a patient-specific surface model of the proximal femur from sparse input data, which may consist of sparse point data or a limited number of calibrated X-ray images. Depending on the modality of the input data, our approach chooses different PDMs. When 3D sparse points are used, which may be obtained intraoperatively via a pointer-based digitization or from a calibrated ultrasound, a fine level point distribution model (FL-PDM) is used in the reconstruction process. In contrast, when calibrated X-ray images are used, which may be obtained preoperatively or intraoperatively, a coarse level point distribution model (CL-PDM) will be used. Results: The present approach was verified on 31 femurs. Three different types of input data, i.e., sparse points, calibrated fluoroscopic images, and calibrated X-ray radiographs, were used in our experiments to reconstruct a surface model of the associated bone. Our experimental results demonstrate promising accuracy of the present approach. Conclusions: A multi-resolution point distribution model facilitate the reconstruction of a patient-specific surface model of the proximal femur from sparse input dat
3D SEM Surface Reconstruction from Multi-View Images
The scanning electron microscope (SEM), a promising imaging equipment has been used to determine the surface properties such as compositions or geometries of specimens by achieving increased magnification, contrast, and resolution. SEM micro-graphs, however, remain two-dimensional (2D). The knowledge and information about their three-dimensional (3D) surface structures are critical in many real-world applications. Having 3D surfaces from SEM images provides true anatomic shapes of micro-scale samples which allow for quantitative measurements and informative visualization of the systems being investigated. A novel multi-view approach for reconstruction of SEM images is demonstrated in this research project. This thesis focuses on the 3D SEM surface reconstruction from multi-view images. We investigate an approach to reconstruction of 3D surfaces from stereo SEM image pairs and then discuss how 3D point clouds may be registered to generate more complete 3D shapes from multi-views of the microscopic specimen. Then we introduce a method that uses an algorithm called KAZE, which reconstructs 3D surfaces from multiple views of objects. Then Numerous results are presented to show the effectiveness of the presented approaches
Neuralangelo: High-Fidelity Neural Surface Reconstruction
Neural surface reconstruction has been shown to be powerful for recovering
dense 3D surfaces via image-based neural rendering. However, current methods
struggle to recover detailed structures of real-world scenes. To address the
issue, we present Neuralangelo, which combines the representation power of
multi-resolution 3D hash grids with neural surface rendering. Two key
ingredients enable our approach: (1) numerical gradients for computing
higher-order derivatives as a smoothing operation and (2) coarse-to-fine
optimization on the hash grids controlling different levels of details. Even
without auxiliary inputs such as depth, Neuralangelo can effectively recover
dense 3D surface structures from multi-view images with fidelity significantly
surpassing previous methods, enabling detailed large-scale scene reconstruction
from RGB video captures.Comment: CVPR 2023, project page:
https://research.nvidia.com/labs/dir/neuralangel
Scene Reconstruction from Multi-Scale Input Data
Geometry acquisition of real-world objects by means of 3D scanning or stereo reconstruction constitutes a very important and challenging problem in computer vision. 3D scanners and stereo algorithms usually provide geometry from one viewpoint only, and several of the these scans need to be merged into one consistent representation. Scanner data generally has lower noise levels than stereo methods and the scanning scenario is more controlled. In image-based stereo approaches, the aim is to reconstruct the 3D surface of an object solely from multiple photos of the object. In many cases, the stereo geometry is contaminated with noise and outliers, and exhibits large variations in scale. Approaches that fuse such data into one consistent surface must be resilient to such imperfections.
In this thesis, we take a closer look at geometry reconstruction using both scanner data and the more challenging image-based scene reconstruction approaches. In particular, this work focuses on the uncontrolled setting where the input images are not constrained, may be taken with different camera models, under different lighting and weather conditions, and from vastly different points of view. A typical dataset contains many views that observe the scene from an overview perspective, and relatively few views capture small details of the geometry. What results from these datasets are surface samples of the scene with vastly different resolution. As we will show in this thesis, the multi-resolution, or, "multi-scale" nature of the input is a relevant aspect for surface reconstruction, which has rarely been considered in literature yet. Integrating scale as additional information in the reconstruction process can make a substantial difference in surface quality.
We develop and study two different approaches for surface reconstruction that are able to cope with the challenges resulting from uncontrolled images. The first approach implements surface reconstruction by fusion of depth maps using a multi-scale hierarchical signed distance function. The hierarchical representation allows fusion of multi-resolution depth maps without mixing geometric information at incompatible scales, which preserves detail in high-resolution regions. An incomplete octree is constructed by incrementally adding triangulated depth maps to the hierarchy, which leads to scattered samples of the multi-resolution signed distance function. A continuous representation of the scattered data is defined by constructing a tetrahedral complex, and a final, highly-adaptive surface is extracted by applying the Marching Tetrahedra algorithm.
A second, point-based approach is based on a more abstract, multi-scale implicit function defined as a sum of basis functions. Each input sample contributes a single basis function which is parameterized solely by the sample's attributes, effectively yielding a parameter-free method. Because the scale of each sample controls the size of the basis function, the method automatically adapts to data redundancy for noise reduction and is highly resilient to the quality-degrading effects of low-resolution samples, thus favoring high-resolution surfaces.
Furthermore, we present a robust, image-based reconstruction system for surface modeling: MVE, the Multi-View Environment. The implementation provides all steps involved in the pipeline: Calibration and registration of the input images, dense geometry reconstruction by means of stereo, a surface reconstruction step and post-processing, such as remeshing and texturing. In contrast to other software solutions for image-based reconstruction, MVE handles large, uncontrolled, multi-scale datasets as well as input from more controlled capture scenarios. The reason lies in the particular choice of the multi-view stereo and surface reconstruction algorithms.
The resulting surfaces are represented using a triangular mesh, which is a piecewise linear approximation to the real surface. The individual triangles are often so small that they barely contribute any geometric information and can be ill-shaped, which can cause numerical problems. A surface remeshing approach is introduced which changes the surface discretization such that more favorable triangles are created. It distributes the vertices of the mesh according to a density function, which is derived from the curvature of the geometry. Such a mesh is better suited for further processing and has reduced storage requirements.
We thoroughly compare the developed methods against the state-of-the art and also perform a qualitative evaluation of the two surface reconstruction methods on a wide range of datasets with different properties. The usefulness of the remeshing approach is demonstrated on both scanner and multi-view stereo data
- …