3D object reconstruction from image sequences with a one line search method

Session 4 - Poster Session 1: Motion, 3D, Recognition, Feature: no. 4-20This paper addresses the problem of estimating the 3D model of an object from a sequence of images where the object is visible from different point of views. In particular, the paper considers the case of turntable image sequences, i.e. images captured under circular motion. A new method is hence proposed for this problem, which consists of determining the image point correspondences of each 3D point through a one line search where a photo-consistency index is maximized. Results with both synthetic and real data validate and illustrate the proposed method.postprintThe IAPR Conference on Machine Vision Applications (MVA2011), Nara, Japan, 13-15 June 2011. In Proceedings of MVA2011, 2011, p. 108-11

Multiview Stereo Object Reconstruction with a One-Line Search Method

published_or_final_versio

Extracting surface representations from rim curves

LNCS v. 3852 is the conference proceedings of ACCV 2006In this paper, we design and implement a novel method for constructing a mixed triangle/quadrangle mesh from the 3D space curves (rims) estimated from the profiles of an object in an image sequence without knowing the original 3D topology of the object. To this aim, a contour data structure for representing visual hull, which is different from that for CT/MRI, is introduced. In this paper, we (1) solve the "branching structure" problem by introducing some additional "directed edge", and (2) extract a triangle/ quadrangle closed mesh from the contour structure with an algorithm based on dynamic programming. Both theoretical demonstration and real world results show that our proposed method has sufficient robustness with respect to the complex topology of the object, and the extracted mesh is of high quality. © Springer-Verlag Berlin Heidelberg 2006.postprintThe 7th Asian Conference on Computer Vision (ACCV 2006), Hyderabad, India, 13-16 January 2006. In Lecture Notes in Computer Science, 2006, v. 3852, p. 732-74

Structure and motion estimation from apparent contours under circular motion

In this paper, we address the problem of recovering structure and motion from the apparent contours of a smooth surface. Fixed image features under circular motion and their relationships with the intrinsic parameters of the camera are exploited to provide a simple parameterization of the fundamental matrix relating any pair of views in the sequence. Such a parameterization allows a trivial initialization of the motion parameters, which all bear physical meaning. It also greatly reduces the dimension of the search space for the optimization problem, which can now be solved using only two epipolar tangents. In contrast to previous methods, the motion estimation algorithm introduced here can cope with incomplete circular motion and more widely spaced images. Existing techniques for model reconstruction from apparent contours are then reviewed and compared. Experiment on real data has been carried out and the 3D model reconstructed from the estimated motion is presented. © 2002 Elsevier Science B.V. All rights reserved.postprin

3D object reconstruction using computer vision : reconstruction and characterization applications for external human anatomical structures

Tese de doutoramento. Engenharia Informática. Faculdade de Engenharia. Universidade do Porto. 201

Robust surface modelling of visual hull from multiple silhouettes

Reconstructing depth information from images is one of the actively researched themes in computer vision and its application involves most vision research areas from object recognition to realistic visualisation. Amongst other useful vision-based reconstruction techniques, this thesis extensively investigates the visual hull (VH) concept for volume approximation and its robust surface modelling when various views of an object are available. Assuming that multiple images are captured from a circular motion, projection matrices are generally parameterised in terms of a rotation angle from a reference position in order to facilitate the multi-camera calibration. However, this assumption is often violated in practice, i.e., a pure rotation in a planar motion with accurate rotation angle is hardly realisable. To address this problem, at first, this thesis proposes a calibration method associated with the approximate circular motion. With these modified projection matrices, a resulting VH is represented by a hierarchical tree structure of voxels from which surfaces are extracted by the Marching cubes (MC) algorithm. However, the surfaces may have unexpected artefacts caused by a coarser volume reconstruction, the topological ambiguity of the MC algorithm, and imperfect image processing or calibration result. To avoid this sensitivity, this thesis proposes a robust surface construction algorithm which initially classifies local convex regions from imperfect MC vertices and then aggregates local surfaces constructed by the 3D convex hull algorithm. Furthermore, this thesis also explores the use of wide baseline images to refine a coarse VH using an affine invariant region descriptor. This improves the quality of VH when a small number of initial views is given. In conclusion, the proposed methods achieve a 3D model with enhanced accuracy. Also, robust surface modelling is retained when silhouette images are degraded by practical noise

3D model reconstruction with noise filtering using boundary edges.

Lau Tak Fu.Thesis submitted in: October 2003.Thesis (M.Phil.)--Chinese University of Hong Kong, 2004.Includes bibliographical references (leaves 93-98).Abstracts in English and Chinese.Chapter 1 - --- Introduction --- p.9Chapter 1.1 --- Scope of the work --- p.9Chapter 1.2 --- Main contribution --- p.11Chapter 1.3 --- Outline of the thesis --- p.12Chapter 2 - --- Background --- p.14Chapter 2.1 --- Three dimensional models from images --- p.14Chapter 2.2 --- Un-calibrated 3D reconstruction --- p.14Chapter 2.3 --- Self calibrated 3D reconstruction --- p.16Chapter 2.4 --- Initial model formation using image based --- p.18Chapter 2.5 --- Volumes from Silhouettes --- p.19Chapter 3 - --- Initial model reconstruct the problem with mismatch noise --- p.22Chapter 3.1 --- Perspective Camera Model --- p.24Chapter 3.2 --- "Intrinsic parameters, Extrinsic parameters and camera motion" --- p.25Chapter 3.2.1 --- Intrinsic parameters --- p.25Chapter 3.2.2 --- Extrinsic parameter and camera motion --- p.27Chapter 3.3 --- Lowe's method --- p.29Chapter 3.4 --- Interleave bundle adjustment for structure and motion recovery from multiple images --- p.32Chapter 3.5 --- Feature points mismatch analysis --- p.38Chapter 4 - --- Feature selection by using look forward silhouette clipping --- p.43Chapter 4.1 --- Introduction to silhouette clipping --- p.43Chapter 4.2 --- Silhouette clipping for 3D model --- p.45Chapter 4.3 --- Implementation --- p.52Chapter 4.3.1 --- Silhouette extraction program --- p.52Chapter 4.3.2 --- Feature filter for alternative bundle adjustment algorithm --- p.59Chapter 5 - --- Experimental data --- p.61Chapter 5.1 --- Simulation --- p.61Chapter 5.1.1 --- Input of simulation --- p.61Chapter 5.1.2 --- Output of the simulation --- p.66Chapter 5.1.2.1 --- Radius distribution --- p.66Chapter 5.1.2.2 --- 3D model output --- p.74Chapter 5.1.2.3 --- VRML plotting --- p.80Chapter 5.2 --- Real Image testing --- p.82Chapter 5.2.1 --- Toy house on a turntable test --- p.82Chapter 5.2.2 --- Other tests on turntable --- p.86Chapter 6 - --- Conclusion and discussion --- p.8

Robust surface modelling of visual hull from multiple silhouettes

Reconstructing depth information from images is one of the actively researched themes in computer vision and its application involves most vision research areas from object recognition to realistic visualisation. Amongst other useful vision-based reconstruction techniques, this thesis extensively investigates the visual hull (VH) concept for volume approximation and its robust surface modelling when various views of an object are available. Assuming that multiple images are captured from a circular motion, projection matrices are generally parameterised in terms of a rotation angle from a reference position in order to facilitate the multi-camera calibration. However, this assumption is often violated in practice, i.e., a pure rotation in a planar motion with accurate rotation angle is hardly realisable. To address this problem, at first, this thesis proposes a calibration method associated with the approximate circular motion. With these modified projection matrices, a resulting VH is represented by a hierarchical tree structure of voxels from which surfaces are extracted by the Marching cubes (MC) algorithm. However, the surfaces may have unexpected artefacts caused by a coarser volume reconstruction, the topological ambiguity of the MC algorithm, and imperfect image processing or calibration result. To avoid this sensitivity, this thesis proposes a robust surface construction algorithm which initially classifies local convex regions from imperfect MC vertices and then aggregates local surfaces constructed by the 3D convex hull algorithm. Furthermore, this thesis also explores the use of wide baseline images to refine a coarse VH using an affine invariant region descriptor. This improves the quality of VH when a small number of initial views is given. In conclusion, the proposed methods achieve a 3D model with enhanced accuracy. Also, robust surface modelling is retained when silhouette images are degraded by practical noise

Capturing Hand-Object Interaction and Reconstruction of Manipulated Objects

Hand motion capture with an RGB-D sensor gained recently a lot of research attention, however, even most recent approaches focus on the case of a single isolated hand. We focus instead on hands that interact with other hands or with a rigid or articulated object. Our framework successfully captures motion in such scenarios by combining a generative model with discriminatively trained salient points, collision detection and physics simulation to achieve a low tracking error with physically plausible poses. All components are unified in a single objective function that can be optimized with standard optimization techniques. We initially assume a-priori knowledge of the object’s shape and skeleton. In case of unknown object shape there are existing 3d reconstruction methods that capitalize on distinctive geometric or texture features. These methods though fail for textureless and highly symmetric objects like household articles, mechanical parts or toys. We show that extracting 3d hand motion for in-hand scanning e↵ectively facilitates the reconstruction of such objects and we fuse the rich additional information of hands into a 3d reconstruction pipeline. Finally, although shape reconstruction is enough for rigid objects, there is a lack of tools that build rigged models of articulated objects that deform realistically using RGB-D data. We propose a method that creates a fully rigged model consisting of a watertight mesh, embedded skeleton and skinning weights by employing a combination of deformable mesh tracking, motion segmentation based on spectral clustering and skeletonization based on mean curvature flow