12,876 research outputs found
MP-MVS: Multi-Scale Windows PatchMatch and Planar Prior Multi-View Stereo
Significant strides have been made in enhancing the accuracy of Multi-View
Stereo (MVS)-based 3D reconstruction. However, untextured areas with unstable
photometric consistency often remain incompletely reconstructed. In this paper,
we propose a resilient and effective multi-view stereo approach (MP-MVS). We
design a multi-scale windows PatchMatch (mPM) to obtain reliable depth of
untextured areas. In contrast with other multi-scale approaches, which is
faster and can be easily extended to PatchMatch-based MVS approaches.
Subsequently, we improve the existing checkerboard sampling schemes by limiting
our sampling to distant regions, which can effectively improve the efficiency
of spatial propagation while mitigating outlier generation. Finally, we
introduce and improve planar prior assisted PatchMatch of ACMP. Instead of
relying on photometric consistency, we utilize geometric consistency
information between multi-views to select reliable triangulated vertices. This
strategy can obtain a more accurate planar prior model to rectify photometric
consistency measurements. Our approach has been tested on the ETH3D High-res
multi-view benchmark with several state-of-the-art approaches. The results
demonstrate that our approach can reach the state-of-the-art. The associated
codes will be accessible at https://github.com/RongxuanTan/MP-MVS
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
Depth Assisted Full Resolution Network for Single Image-based View Synthesis
Researches in novel viewpoint synthesis majorly focus on interpolation from
multi-view input images. In this paper, we focus on a more challenging and
ill-posed problem that is to synthesize novel viewpoints from one single input
image. To achieve this goal, we propose a novel deep learning-based technique.
We design a full resolution network that extracts local image features with the
same resolution of the input, which contributes to derive high resolution and
prevent blurry artifacts in the final synthesized images. We also involve a
pre-trained depth estimation network into our system, and thus 3D information
is able to be utilized to infer the flow field between the input and the target
image. Since the depth network is trained by depth order information between
arbitrary pairs of points in the scene, global image features are also involved
into our system. Finally, a synthesis layer is used to not only warp the
observed pixels to the desired positions but also hallucinate the missing
pixels with recorded pixels. Experiments show that our technique performs well
on images of various scenes, and outperforms the state-of-the-art techniques
Unsupervised Monocular Depth Estimation with Left-Right Consistency
Learning based methods have shown very promising results for the task of
depth estimation in single images. However, most existing approaches treat
depth prediction as a supervised regression problem and as a result, require
vast quantities of corresponding ground truth depth data for training. Just
recording quality depth data in a range of environments is a challenging
problem. In this paper, we innovate beyond existing approaches, replacing the
use of explicit depth data during training with easier-to-obtain binocular
stereo footage.
We propose a novel training objective that enables our convolutional neural
network to learn to perform single image depth estimation, despite the absence
of ground truth depth data. Exploiting epipolar geometry constraints, we
generate disparity images by training our network with an image reconstruction
loss. We show that solving for image reconstruction alone results in poor
quality depth images. To overcome this problem, we propose a novel training
loss that enforces consistency between the disparities produced relative to
both the left and right images, leading to improved performance and robustness
compared to existing approaches. Our method produces state of the art results
for monocular depth estimation on the KITTI driving dataset, even outperforming
supervised methods that have been trained with ground truth depth.Comment: CVPR 2017 ora
- …