417 research outputs found
Building with Drones: Accurate 3D Facade Reconstruction using MAVs
Automatic reconstruction of 3D models from images using multi-view
Structure-from-Motion methods has been one of the most fruitful outcomes of
computer vision. These advances combined with the growing popularity of Micro
Aerial Vehicles as an autonomous imaging platform, have made 3D vision tools
ubiquitous for large number of Architecture, Engineering and Construction
applications among audiences, mostly unskilled in computer vision. However, to
obtain high-resolution and accurate reconstructions from a large-scale object
using SfM, there are many critical constraints on the quality of image data,
which often become sources of inaccuracy as the current 3D reconstruction
pipelines do not facilitate the users to determine the fidelity of input data
during the image acquisition. In this paper, we present and advocate a
closed-loop interactive approach that performs incremental reconstruction in
real-time and gives users an online feedback about the quality parameters like
Ground Sampling Distance (GSD), image redundancy, etc on a surface mesh. We
also propose a novel multi-scale camera network design to prevent scene drift
caused by incremental map building, and release the first multi-scale image
sequence dataset as a benchmark. Further, we evaluate our system on real
outdoor scenes, and show that our interactive pipeline combined with a
multi-scale camera network approach provides compelling accuracy in multi-view
reconstruction tasks when compared against the state-of-the-art methods.Comment: 8 Pages, 2015 IEEE International Conference on Robotics and
Automation (ICRA '15), Seattle, WA, US
PetroSurf3D - A Dataset for high-resolution 3D Surface Segmentation
The development of powerful 3D scanning hardware and reconstruction
algorithms has strongly promoted the generation of 3D surface reconstructions
in different domains. An area of special interest for such 3D reconstructions
is the cultural heritage domain, where surface reconstructions are generated to
digitally preserve historical artifacts. While reconstruction quality nowadays
is sufficient in many cases, the robust analysis (e.g. segmentation, matching,
and classification) of reconstructed 3D data is still an open topic. In this
paper, we target the automatic and interactive segmentation of high-resolution
3D surface reconstructions from the archaeological domain. To foster research
in this field, we introduce a fully annotated and publicly available
large-scale 3D surface dataset including high-resolution meshes, depth maps and
point clouds as a novel benchmark dataset to the community. We provide baseline
results for our existing random forest-based approach and for the first time
investigate segmentation with convolutional neural networks (CNNs) on the data.
Results show that both approaches have complementary strengths and weaknesses
and that the provided dataset represents a challenge for future research.Comment: CBMI Submission; Dataset and more information can be found at
http://lrs.icg.tugraz.at/research/petroglyphsegmentation
Estimation incrémentale de surface à partir d'un nuage de point épars reconstruit à partir d'images omnidirectionnelles
National audienceCet article introduit une méthode incrémentale de reconstruction de surface (une 2-variété). Elle prend en entrée un nuage de points 3D épars reconstruit à partir d'une séquence d'images, par opposition aux algorithmes habituels denses. De plus, notre méthode est incrémentale : la surface est mise à jour à chaque nouvelle pose de caméra donnée en entrée, et la mise à jour a lieu dans un voisinage restreint de la nouvelle pose. Comparée aux autres méthodes de reconstruction de surface, notre méthode a l'avantage de cumuler toutes ces propriétés (nuage épars en entrée, 2-variété en sortie, calcul incrémental et local). La qualité et le temps d'exécution sont évalués sur une séquence d'images omnidirectionnelles (longue de 2.5 km) prise en environnement urbain, et la méthode est quantitativement évaluée sur une séquence urbaine synthétique
A Comparative Neural Radiance Field (NeRF) 3D Analysis of Camera Poses from HoloLens Trajectories and Structure from Motion
Neural Radiance Fields (NeRFs) are trained using a set of camera poses and
associated images as input to estimate density and color values for each
position. The position-dependent density learning is of particular interest for
photogrammetry, enabling 3D reconstruction by querying and filtering the NeRF
coordinate system based on the object density. While traditional methods like
Structure from Motion are commonly used for camera pose calculation in
pre-processing for NeRFs, the HoloLens offers an interesting interface for
extracting the required input data directly. We present a workflow for
high-resolution 3D reconstructions almost directly from HoloLens data using
NeRFs. Thereby, different investigations are considered: Internal camera poses
from the HoloLens trajectory via a server application, and external camera
poses from Structure from Motion, both with an enhanced variant applied through
pose refinement. Results show that the internal camera poses lead to NeRF
convergence with a PSNR of 25\,dB with a simple rotation around the x-axis and
enable a 3D reconstruction. Pose refinement enables comparable quality compared
to external camera poses, resulting in improved training process with a PSNR of
27\,dB and a better 3D reconstruction. Overall, NeRF reconstructions outperform
the conventional photogrammetric dense reconstruction using Multi-View Stereo
in terms of completeness and level of detail.Comment: 7 pages, 5 figures. Will be published in the ISPRS The International
Archives of Photogrammetry, Remote Sensing and Spatial Information Science
Level-SfM: Structure from Motion on Neural Level Set of Implicit Surfaces
This paper presents a neural incremental Structure-from-Motion (SfM)
approach, Level-SfM. In our formulation, we aim at simultaneously learning
coordinate MLPs for the implicit surfaces and the radiance fields, and
estimating the camera poses and scene geometry, which is mainly sourced from
the established keypoint correspondences by SIFT. Our formulation would face
some new challenges due to inevitable two-view and few-view configurations at
the beginning of incremental SfM pipeline for the optimization of coordinate
MLPs, but we found that the strong inductive biases conveying in the 2D
correspondences are feasible and promising to avoid those challenges by
exploiting the relationship between the ray sampling schemes used in volumetric
rendering and the sphere tracing of finding the zero-level set of implicit
surfaces. Based on this, we revisit the pipeline of incremental SfM and renew
the key components of two-view geometry initialization, the camera pose
registration, and the 3D points triangulation, as well as the Bundle Adjustment
in a novel perspective of neural implicit surfaces. Because the coordinate MLPs
unified the scene geometry in small MLP networks, our Level-SfM treats the
zero-level set of the implicit surface as an informative top-down
regularization to manage the reconstructed 3D points, reject the outlier of
correspondences by querying SDF, adjust the estimated geometries by NBA (Neural
BA), finally yielding promising results of 3D reconstruction. Furthermore, our
Level-SfM alleviated the requirement of camera poses for neural 3D
reconstruction.Comment: under revie
Towards Live 3D Reconstruction from Wearable Video: An Evaluation of V-SLAM, NeRF, and Videogrammetry Techniques
Mixed reality (MR) is a key technology which promises to change the future of
warfare. An MR hybrid of physical outdoor environments and virtual military
training will enable engagements with long distance enemies, both real and
simulated. To enable this technology, a large-scale 3D model of a physical
environment must be maintained based on live sensor observations. 3D
reconstruction algorithms should utilize the low cost and pervasiveness of
video camera sensors, from both overhead and soldier-level perspectives.
Mapping speed and 3D quality can be balanced to enable live MR training in
dynamic environments. Given these requirements, we survey several 3D
reconstruction algorithms for large-scale mapping for military applications
given only live video. We measure 3D reconstruction performance from common
structure from motion, visual-SLAM, and photogrammetry techniques. This
includes the open source algorithms COLMAP, ORB-SLAM3, and NeRF using
Instant-NGP. We utilize the autonomous driving academic benchmark KITTI, which
includes both dashboard camera video and lidar produced 3D ground truth. With
the KITTI data, our primary contribution is a quantitative evaluation of 3D
reconstruction computational speed when considering live video.Comment: Accepted to 2022 Interservice/Industry Training, Simulation, and
Education Conference (I/ITSEC), 13 page
TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Rendering
We present a new pipeline for acquiring a textured mesh in the wild with a
single smartphone which offers access to images, depth maps, and valid poses.
Our method first introduces an RGBD-aided structure from motion, which can
yield filtered depth maps and refines camera poses guided by corresponding
depth. Then, we adopt the neural implicit surface reconstruction method, which
allows for high-quality mesh and develops a new training process for applying a
regularization provided by classical multi-view stereo methods. Moreover, we
apply a differentiable rendering to fine-tune incomplete texture maps and
generate textures which are perceptually closer to the original scene. Our
pipeline can be applied to any common objects in the real world without the
need for either in-the-lab environments or accurate mask images. We demonstrate
results of captured objects with complex shapes and validate our method
numerically against existing 3D reconstruction and texture mapping methods.Comment: Accepted to CVPR23. Project Page: https://jh-choi.github.io/TMO
Cloud-based collaborative 3D reconstruction using smartphones
This article presents a pipeline that enables multiple users to collaboratively acquire images with monocular smartphones and derive a 3D point cloud using a remote reconstruction server. A set of key images are automatically selected from each smartphone’s camera video feed as multiple users record different viewpoints of an object, concurrently or at different time instants. Selected images are automatically processed and registered with an incremental Structure from Motion (SfM) algorithm in order to create a 3D model. Our incremental SfM approach enables on-the- y feedback to the user to be generated about current reconstruction progress. Feedback is provided in the form of a preview window showing the current 3D point cloud, enabling users to see if parts of a surveyed scene need further attention/coverage whilst they are still in situ. We evaluate our 3D reconstruction pipeline by performing experiments in uncontrolled and unconstrained real-world scenarios. Datasets are publicly available
- …