Trifocal Relative Pose from Lines at Points and its Efficient Solution

We present a new minimal problem for relative pose estimation mixing point features with lines incident at points observed in three views and its efficient homotopy continuation solver. We demonstrate the generality of the approach by analyzing and solving an additional problem with mixed point and line correspondences in three views. The minimal problems include correspondences of (i) three points and one line and (ii) three points and two lines through two of the points which is reported and analyzed here for the first time. These are difficult to solve, as they have 216 and - as shown here - 312 solutions, but cover important practical situations when line and point features appear together, e.g., in urban scenes or when observing curves. We demonstrate that even such difficult problems can be solved robustly using a suitable homotopy continuation technique and we provide an implementation optimized for minimal problems that can be integrated into engineering applications. Our simulated and real experiments demonstrate our solvers in the camera geometry computation task in structure from motion. We show that new solvers allow for reconstructing challenging scenes where the standard two-view initialization of structure from motion fails.Comment: This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while most authors were in residence at Brown University's Institute for Computational and Experimental Research in Mathematics -- ICERM, in Providence, R

Image-Based View Synthesis

We present a new method for rendering novel images of flexible 3D objects from a small number of example images in correspondence. The strength of the method is the ability to synthesize images whose viewing position is significantly far away from the viewing cone of the example images ("view extrapolation"), yet without ever modeling the 3D structure of the scene. The method relies on synthesizing a chain of "trilinear tensors" that governs the warping function from the example images to the novel image, together with a multi-dimensional interpolation function that synthesizes the non-rigid motions of the viewed object from the virtual camera position. We show that two closely spaced example images alone are sufficient in practice to synthesize a significant viewing cone, thus demonstrating the ability of representing an object by a relatively small number of model images --- for the purpose of cheap and fast viewers that can run on standard hardware

On Degeneracy of Linear Reconstruction from Three Views: Linear Line Complex and Applications

This paper investigates the linear degeneracies of projective structure estimation from point and line features across three views. We show that the rank of the linear system of equations for recovering the trilinear tensor of three views reduces to 23 (instead of 26) in the case when the scene is a Linear Line Complex (set of lines in space intersecting at a common line) and is 21 when the scene is planar. The LLC situation is only linearly degenerate, and we show that one can obtain a unique solution when the admissibility constraints of the tensor are accounted for. The line configuration described by an LLC, rather than being some obscure case, is in fact quite typical. It includes, as a particular example, the case of a camera moving down a hallway in an office environment or down an urban street. Furthermore, an LLC situation may occur as an artifact such as in direct estimation from spatio-temporal derivatives of image brightness. Therefore, an investigation into degeneracies and their remedy is important also in practice

A Self-calibration Algorithm Based on a Unified Framework for Constraints on Multiple Views

In this paper, we propose a new self-calibration algorithm for upgrading projective space to Euclidean space. The proposed method aims to combine the most commonly used metric constraints, including zero skew and unit aspect-ratio by formulating each constraint as a cost function within a unified framework. Additional constraints, e.g., constant principal points, can also be formulated in the same framework. The cost function is very flexible and can be composed of different constraints on different views. The upgrade process is then stated as a minimization problem which may be solved by minimizing an upper bound of the cost function. This proposed method is non-iterative. Experimental results on synthetic data and real data are presented to show the performance of the proposed method and accuracy of the reconstructed scene. © 2012 The Author(s).published_or_final_versionSpringer Open Choice, 25 May 201

Inherent Two-Way Ambiguity in 2D Projective Reconstruction from Three Uncalibrated 1D Images

International audienceIt is shown that there always exists a two-way ambiguity for 2D projective reconstruction from three uncalibrated 1D views independent of the number of point correspondences. It is also shown that the two distinct projective reconstructions are exactly related by a quadratic transformation with the three camera centers as the fundamental points. The unique reconstruction exists only for the case where the three camera centers are aligned. The theoretical results are demonstrated on numerical examples

Towards automated capture of 3D foot geometry for custom orthoses

This thesis presents a novel method of capturing 3D foot geometry from images for custom shoe insole manufacture. Orthopedic footwear plays an important role as a treatment and prevention of foot conditions associated with diabetes. Through the use of customized shoe insoles, a podiatrist can provide a means to better distribute the pressure around the foot, and can also correct the biomechanics of the foot. Different foot scanners are used to obtain the geometric plantar surface of foot, but are expensive and more generic in nature. The focus of this thesis is to build 3D foot structure from a pair of calibrated images. The process begins with considering a pair of good images of the foot, obtained from the scanner utility frame. The next step involves identifying corners or features in the images. Correlation between the selected features forms the fundamental part of epipolar analysis. Rigorous techniques are implemented for robust feature matching. A 3D point cloud is then obtained by applying the 8-point algorithm and linear 3D triangulation method. The advantage of this system is quick capture of foot geometry and minimal intervention from the user. A reconstructed 3D point cloud of foot is presented to verify this method as inexpensive and more suited to the needs of the podiatrist

Image motion estimation for 3D model based video conferencing.

Cheung Man-kin.Thesis (M.Phil.)--Chinese University of Hong Kong, 2000.Includes bibliographical references (leaves 116-120).Abstracts in English and Chinese.Chapter 1) --- Introduction --- p.1Chapter 1.1) --- Building of the 3D Wireframe and Facial Model --- p.2Chapter 1.2) --- Description of 3D Model Based Video Conferencing --- p.3Chapter 1.3) --- Wireframe Model Fitting or Conformation --- p.6Chapter 1.4) --- Pose Estimation --- p.8Chapter 1.5) --- Facial Motion Estimation and Synthesis --- p.9Chapter 1.6) --- Thesis Outline --- p.10Chapter 2) --- Wireframe model Fitting --- p.11Chapter 2.1) --- Algorithm of WFM Fitting --- p.12Chapter 2.1.1) --- Global Deformation --- p.14Chapter a) --- Scaling --- p.14Chapter b) --- Shifting --- p.15Chapter 2.1.2) --- Local Deformation --- p.15Chapter a) --- Shifting --- p.16Chapter b) --- Scaling --- p.17Chapter 2.1.3) --- Fine Updating --- p.17Chapter 2.2) --- Steps of Fitting --- p.18Chapter 2.3) --- Functions of Different Deformation --- p.18Chapter 2.4) --- Experimental Results --- p.19Chapter 2.4.1) --- Output wireframe in each step --- p.19Chapter 2.4.2) --- Examples of Mis-fitted wireframe with incoming image --- p.22Chapter 2.4.3) --- Fitted 3D facial wireframe --- p.23Chapter 2.4.4) --- Effect of mis-fitted wireframe after compensation of motion --- p.24Chapter 2.5) --- Summary --- p.26Chapter 3) --- Epipolar Geometry --- p.27Chapter 3.1) --- Pinhole Camera Model and Perspective Projection --- p.28Chapter 3.2) --- Concepts in Epipolar Geometry --- p.31Chapter 3.2.1) --- Working with normalized image coordinates --- p.33Chapter 3.2.2) --- Working with pixel image coordinates --- p.35Chapter 3.2.3) --- Summary --- p.37Chapter 3.3) --- 8-point Algorithm (Essential and Fundamental Matrix) --- p.38Chapter 3.3.1) --- Outline of the 8-point algorithm --- p.38Chapter 3.3.2) --- Modification on obtained Fundamental Matrix --- p.39Chapter 3.3.3) --- Transformation of Image Coordinates --- p.40Chapter a) --- Translation to mean of points --- p.40Chapter b) --- Normalizing transformation --- p.41Chapter 3.3.4) --- Summary of 8-point algorithm --- p.41Chapter 3.4) --- Estimation of Object Position by Decomposition of Essential Matrix --- p.43Chapter 3.4.1) --- Algorithm Derivation --- p.43Chapter 3.4.2) --- Algorithm Outline --- p.46Chapter 3.5) --- Noise Sensitivity --- p.48Chapter 3.5.1) --- Rotation vector of model --- p.48Chapter 3.5.2) --- The projection of rotated model --- p.49Chapter 3.5.3) --- Noisy image --- p.51Chapter 3.5.4) --- Summary --- p.51Chapter 4) --- Pose Estimation --- p.54Chapter 4.1) --- Linear Method --- p.55Chapter 4.1.1) --- Theory --- p.55Chapter 4.1.2) --- Normalization --- p.57Chapter 4.1.3) --- Experimental Results --- p.58Chapter a) --- Synthesized image by linear method without normalization --- p.58Chapter b) --- Performance between linear method with and without normalization --- p.60Chapter c) --- Performance of linear method under quantization noise with different transformation components --- p.62Chapter d) --- Performance of normalized case without transformation in z- component --- p.63Chapter 4.1.4) --- Summary --- p.64Chapter 4.2) --- Two Stage Algorithm --- p.66Chapter 4.2.1) --- Introduction --- p.66Chapter 4.2.2) --- The Two Stage Algorithm --- p.67Chapter a) --- Stage 1 (Iterative Method) --- p.68Chapter b) --- Stage 2 ( Non-linear Optimization) --- p.71Chapter 4.2.3) --- Summary of the Two Stage Algorithm --- p.72Chapter 4.2.4) --- Experimental Results --- p.72Chapter 4.2.5) --- Summary --- p.80Chapter 5) --- Facial Motion Estimation and Synthesis --- p.81Chapter 5.1) --- Facial Expression based on face muscles --- p.83Chapter 5.1.1) --- Review of Action Unit Approach --- p.83Chapter 5.1.2) --- Distribution of Motion Unit --- p.85Chapter 5.1.3) --- Algorithm --- p.89Chapter a) --- For Unidirectional Motion Unit --- p.89Chapter b) --- For Circular Motion Unit (eyes) --- p.90Chapter c) --- For Another Circular Motion Unit (mouth) --- p.90Chapter 5.1.4) --- Experimental Results --- p.91Chapter 5.1.5) --- Summary --- p.95Chapter 5.2) --- Detection of Facial Expression by Muscle-based Approach --- p.96Chapter 5.2.1) --- Theory --- p.96Chapter 5.2.2) --- Algorithm --- p.97Chapter a) --- For Sheet Muscle --- p.97Chapter b) --- For Circular Muscle --- p.98Chapter c) --- For Mouth Muscle --- p.99Chapter 5.2.3) --- Steps of Algorithm --- p.100Chapter 5.2.4) --- Experimental Results --- p.101Chapter 5.2.5) --- Summary --- p.103Chapter 6) --- Conclusion --- p.104Chapter 6.1) --- WFM fitting --- p.104Chapter 6.2) --- Pose Estimation --- p.105Chapter 6.3) --- Facial Estimation and Synthesis --- p.106Chapter 6.4) --- Discussion on Future Improvements --- p.107Chapter 6.4.1) --- WFM Fitting --- p.107Chapter 6.4.2) --- Pose Estimation --- p.109Chapter 6.4.3) --- Facial Motion Estimation and Synthesis --- p.110Chapter 7) --- Appendix --- p.111Chapter 7.1) --- Newton's Method or Newton-Raphson Method --- p.111Chapter 7.2) --- H.261 --- p.113Chapter 7.3) --- 3D Measurement --- p.114Bibliography --- p.11