Cascaded Scene Flow Prediction using Semantic Segmentation

Given two consecutive frames from a pair of stereo cameras, 3D scene flow methods simultaneously estimate the 3D geometry and motion of the observed scene. Many existing approaches use superpixels for regularization, but may predict inconsistent shapes and motions inside rigidly moving objects. We instead assume that scenes consist of foreground objects rigidly moving in front of a static background, and use semantic cues to produce pixel-accurate scene flow estimates. Our cascaded classification framework accurately models 3D scenes by iteratively refining semantic segmentation masks, stereo correspondences, 3D rigid motion estimates, and optical flow fields. We evaluate our method on the challenging KITTI autonomous driving benchmark, and show that accounting for the motion of segmented vehicles leads to state-of-the-art performance.Comment: International Conference on 3D Vision (3DV), 2017 (oral presentation

Hierarchical shape-based surface reconstruction for dense multi-view stereo

International audienceThe recent widespread availability of urban imagery has lead to a growing demand for automatic modeling from multiple images. However, modern image-based modeling research has focused either on highly detailed reconstructions of mostly small objects or on human-assisted simplified modeling. This paper presents a novel algorithm which automatically outputs a simplified, segmented model of a scene from a set of calibrated input images, capturing its essential geometric features. Our approach combines three successive steps. First, a dense point cloud is created from sparse depth maps computed from the input images. Then, shapes are robustly extracted from this set of points. Finally, a compact model of the scene is built from a spatial subdivision induced by these structures: this model is a global minimum of an energy accounting for the visibility of the final surface. The effectiveness of our method is demonstrated through several results on both synthetic and real data sets, illustrating the various benefits of our algorithm, its robustness and its relevance for architectural scenes

Rate-Distortion Efficient Piecewise Planar 3D Scene Representation from 2-D Images

Cataloged from PDF version of article.In any practical application of the 2-D-to-3-D conversion that involves storage and transmission, representation effi- ciency has an undisputable importance that is not reflected in the attention the topic received. In order to address this problem, a novel algorithm, which yields efficient 3-D representations in the rate distortion sense, is proposed. The algorithm utilizes two views of a scene to build a mesh-based representation incrementally, via adding new vertices, while minimizing a distortion measure. The experimental results indicate that, in scenes that can be approximated by planes, the proposed algorithm is superior to the dense depth map and, in some practical situations, to the block motion vector-based representations in the rate-distortion sense

Cavlectometry: Towards Holistic Reconstruction of Large Mirror Objects

We introduce a method based on the deflectometry principle for the reconstruction of specular objects exhibiting significant size and geometric complexity. A key feature of our approach is the deployment of an Automatic Virtual Environment (CAVE) as pattern generator. To unfold the full power of this extraordinary experimental setup, an optical encoding scheme is developed which accounts for the distinctive topology of the CAVE. Furthermore, we devise an algorithm for detecting the object of interest in raw deflectometric images. The segmented foreground is used for single-view reconstruction, the background for estimation of the camera pose, necessary for calibrating the sensor system. Experiments suggest a significant gain of coverage in single measurements compared to previous methods. To facilitate research on specular surface reconstruction, we will make our data set publicly available