3D object reconstruction from image sequences with a one line search method

Session 4 - Poster Session 1: Motion, 3D, Recognition, Feature: no. 4-20This paper addresses the problem of estimating the 3D model of an object from a sequence of images where the object is visible from different point of views. In particular, the paper considers the case of turntable image sequences, i.e. images captured under circular motion. A new method is hence proposed for this problem, which consists of determining the image point correspondences of each 3D point through a one line search where a photo-consistency index is maximized. Results with both synthetic and real data validate and illustrate the proposed method.postprintThe IAPR Conference on Machine Vision Applications (MVA2011), Nara, Japan, 13-15 June 2011. In Proceedings of MVA2011, 2011, p. 108-11

Multi-view urban scene reconstruction in non-uniform volume

This paper presents a new fully automatic approach for multi-view urban scene reconstruction. Our algorithm is based on the Manhattan-World assumption, which can provide compact models while preserving fidelity of synthetic architectures. Starting from a dense point cloud, we extract its main axes by global optimization, and construct a nonuniform volume based on them. A graph model is created from volume facets rather than voxels. Appropriate edge weights are defined to ensure the validity and quality of the surface reconstruction. Compared with the common pointcloud- to-model methods, the proposed methodology exploits image information to unveil the real structures of holes in the point cloud. Experiments demonstrate the encouraging performance of the algorithm. © 2013 SPIE

Patch based 3D reconstruction of the liver surface from laparoscopic videos

Mestrado em Engenharia MecânicaEndoscope is commonly used in Laparoscopic surgery. It allows to the doctor to perform the surgery without opening a cavity. Therefore, it allows the surgeon to perform smaller incisions and consequently diminish the risk of infection. An investigation regarding the reconstruction of the liver surface from a sequence of images obtained from an endoscope video has been done by the Vision Institute of TUHH, in cooperation with the Medical Center Hamburg-Eppendorf. The goal of this paper is to optimize that reconstruction. Previous works in the institute allowed, using matlab functions, to obtain a first reconstruction. Having the matrices of the transformations between cameras and the reconstructed points in 3D, the goal is to optimize that using a patch based 3D reconstruction. The method consists of the construction of a patch for every point and trying to find a good normal for the patch. Some tools are used for the optimization like normal cross correlation that is the similarity function used for obtaining the match of the corresponded 2D windows in different cameras, and a nonlinear optimization method to look for the best normal who gives the best match between the windows. The main objective is to evaluate this method and draw conclusions about its utility and viability: can this method be used for the optimization in the case of the endoscope video of a liver surface

Depth determination method for a trailer-truck test-bed

A camera is considered as a good sensing device to obtain visual information from its surrounding environment. In a three dimensional space, a camera has a two dimensional plane. Projection of an object on this plane creates a two dimensional projection and loses information of the third dimension. A single projection is not enough to retrieve the lost third dimensional information about the object. Thus it makes it difficult to use a single camera as a sensing instrument. In this project, we have developed a method that determines the three dimensional information using a single camera. The method utilizes an assumption of the camera being in motion, allowing it to take projections on unaligned camera planes at different positions. A mathematical comparison of these projections gives us a deterministic value of depth of the surroundings from these camera planes --Abstract, page iii

From Multiview Image Curves to 3D Drawings

Reconstructing 3D scenes from multiple views has made impressive strides in recent years, chiefly by correlating isolated feature points, intensity patterns, or curvilinear structures. In the general setting - without controlled acquisition, abundant texture, curves and surfaces following specific models or limiting scene complexity - most methods produce unorganized point clouds, meshes, or voxel representations, with some exceptions producing unorganized clouds of 3D curve fragments. Ideally, many applications require structured representations of curves, surfaces and their spatial relationships. This paper presents a step in this direction by formulating an approach that combines 2D image curves into a collection of 3D curves, with topological connectivity between them represented as a 3D graph. This results in a 3D drawing, which is complementary to surface representations in the same sense as a 3D scaffold complements a tent taut over it. We evaluate our results against truth on synthetic and real datasets.Comment: Expanded ECCV 2016 version with tweaked figures and including an overview of the supplementary material available at multiview-3d-drawing.sourceforge.ne

Second-order Shape Optimization for Geometric Inverse Problems in Vision

We develop a method for optimization in shape spaces, i.e., sets of surfaces modulo re-parametrization. Unlike previously proposed gradient flows, we achieve superlinear convergence rates through a subtle approximation of the shape Hessian, which is generally hard to compute and suffers from a series of degeneracies. Our analysis highlights the role of mean curvature motion in comparison with first-order schemes: instead of surface area, our approach penalizes deformation, either by its Dirichlet energy or total variation. Latter regularizer sparks the development of an alternating direction method of multipliers on triangular meshes. Therein, a conjugate-gradients solver enables us to bypass formation of the Gaussian normal equations appearing in the course of the overall optimization. We combine all of the aforementioned ideas in a versatile geometric variation-regularized Levenberg-Marquardt-type method applicable to a variety of shape functionals, depending on intrinsic properties of the surface such as normal field and curvature as well as its embedding into space. Promising experimental results are reported

Minimizing the Multi-view Stereo Reprojection Error for Triangular Surface Meshes

International audienceThis article proposes a variational multi-view stereo vision method based on meshes for recovering 3D scenes (shape and radiance) from images. Our method is based on generative models and minimizes the reprojection error (difference between the observed images and the images synthesized from the reconstruction). Our contributions are twofold. 1) For the first time, we rigorously compute the gradient of the reprojection error for non smooth surfaces defined by discrete triangular meshes. The gradient correctly takes into account the visibility changes that occur when a surface moves; this forces the contours generated by the reconstructed surface to perfectly match with the apparent contours in the input images. 2) We propose an original modification of the Lambertian model to take into account deviations from the constant brightness assumption without explicitly modelling the reflectance properties of the scene or other photometric phenomena involved by the camera model. Our method is thus able to recover the shape and the diffuse radiance of non Lambertian scenes

Multiview photometric stereo

This paper addresses the problem of obtaining complete, detailed reconstructions of textureless shiny objects. We present an algorithm which uses silhouettes of the object, as well as images obtained under changing illumination conditions. In contrast with previous photometric stereo techniques, ours is not limited to a single viewpoint but produces accurate reconstructions in full 3D. A number of images of the object are obtained from multiple viewpoints, under varying lighting conditions. Starting from the silhouettes, the algorithm recovers camera motion and constructs the object's visual hull. This is then used to recover the illumination and initialize a multiview photometric stereo scheme to obtain a closed surface reconstruction. There are two main contributions in this paper: First, we describe a robust technique to estimate light directions and intensities and, second, we introduce a novel formulation of photometric stereo which combines multiple viewpoints and, hence, allows closed surface reconstructions. The algorithm has been implemented as a practical model acquisition system. Here, a quantitative evaluation of the algorithm on synthetic data is presented together with complete reconstructions of challenging real objects. Finally, we show experimentally how, even in the case of highly textured objects, this technique can greatly improve on correspondence-based multiview stereo results