15,764 research outputs found
A One Stop 3D Target Reconstruction and multilevel Segmentation Method
3D object reconstruction and multilevel segmentation are fundamental to
computer vision research. Existing algorithms usually perform 3D scene
reconstruction and target objects segmentation independently, and the
performance is not fully guaranteed due to the challenge of the 3D
segmentation. Here we propose an open-source one stop 3D target reconstruction
and multilevel segmentation framework (OSTRA), which performs segmentation on
2D images, tracks multiple instances with segmentation labels in the image
sequence, and then reconstructs labelled 3D objects or multiple parts with
Multi-View Stereo (MVS) or RGBD-based 3D reconstruction methods. We extend
object tracking and 3D reconstruction algorithms to support continuous
segmentation labels to leverage the advances in the 2D image segmentation,
especially the Segment-Anything Model (SAM) which uses the pretrained neural
network without additional training for new scenes, for 3D object segmentation.
OSTRA supports most popular 3D object models including point cloud, mesh and
voxel, and achieves high performance for semantic segmentation, instance
segmentation and part segmentation on several 3D datasets. It even surpasses
the manual segmentation in scenes with complex structures and occlusions. Our
method opens up a new avenue for reconstructing 3D targets embedded with rich
multi-scale segmentation information in complex scenes. OSTRA is available from
https://github.com/ganlab/OSTRA
General Dynamic Scene Reconstruction from Multiple View Video
This paper introduces a general approach to dynamic scene reconstruction from
multiple moving cameras without prior knowledge or limiting constraints on the
scene structure, appearance, or illumination. Existing techniques for dynamic
scene reconstruction from multiple wide-baseline camera views primarily focus
on accurate reconstruction in controlled environments, where the cameras are
fixed and calibrated and background is known. These approaches are not robust
for general dynamic scenes captured with sparse moving cameras. Previous
approaches for outdoor dynamic scene reconstruction assume prior knowledge of
the static background appearance and structure. The primary contributions of
this paper are twofold: an automatic method for initial coarse dynamic scene
segmentation and reconstruction without prior knowledge of background
appearance or structure; and a general robust approach for joint segmentation
refinement and dense reconstruction of dynamic scenes from multiple
wide-baseline static or moving cameras. Evaluation is performed on a variety of
indoor and outdoor scenes with cluttered backgrounds and multiple dynamic
non-rigid objects such as people. Comparison with state-of-the-art approaches
demonstrates improved accuracy in both multiple view segmentation and dense
reconstruction. The proposed approach also eliminates the requirement for prior
knowledge of scene structure and appearance
Dynamic Body VSLAM with Semantic Constraints
Image based reconstruction of urban environments is a challenging problem
that deals with optimization of large number of variables, and has several
sources of errors like the presence of dynamic objects. Since most large scale
approaches make the assumption of observing static scenes, dynamic objects are
relegated to the noise modeling section of such systems. This is an approach of
convenience since the RANSAC based framework used to compute most multiview
geometric quantities for static scenes naturally confine dynamic objects to the
class of outlier measurements. However, reconstructing dynamic objects along
with the static environment helps us get a complete picture of an urban
environment. Such understanding can then be used for important robotic tasks
like path planning for autonomous navigation, obstacle tracking and avoidance,
and other areas. In this paper, we propose a system for robust SLAM that works
in both static and dynamic environments. To overcome the challenge of dynamic
objects in the scene, we propose a new model to incorporate semantic
constraints into the reconstruction algorithm. While some of these constraints
are based on multi-layered dense CRFs trained over appearance as well as motion
cues, other proposed constraints can be expressed as additional terms in the
bundle adjustment optimization process that does iterative refinement of 3D
structure and camera / object motion trajectories. We show results on the
challenging KITTI urban dataset for accuracy of motion segmentation and
reconstruction of the trajectory and shape of moving objects relative to ground
truth. We are able to show average relative error reduction by a significant
amount for moving object trajectory reconstruction relative to state-of-the-art
methods like VISO 2, as well as standard bundle adjustment algorithms
- …