3D Reconstruction through Segmentation of Multi-View Image Sequences

We propose what we believe is a new approach to 3D reconstruction through the design of a 3D voxel volume, such that all the image information and camera geometry are embedded into one feature space. By customising the volume to be suitable for segmentation, the key idea that we propose is the recovery of a 3D scene through the use of globally optimal geodesic active contours. We also present an extension to this idea by proposing the novel design of a 4D voxel volume to analyse the stereo motion problem in multi-view image sequences

3D Dynamic Scene Reconstruction from Multi-View Image Sequences

A confirmation report outlining my PhD research plan is presented. The PhD research topic is 3D dynamic scene reconstruction from multiple view image sequences. Chapter 1 describes the motivation and research aims. An overview of the progress in the past year is included. Chapter 2 is a review of volumetric scene reconstruction techniques and Chapter 3 is an in-depth description of my proposed reconstruction method. The theory behind the proposed volumetric scene reconstruction method is also presented, including topics in projective geometry, camera calibration and energy minimization. Chapter 4 presents the research plan and outlines the future work planned for the next two years

Structure from motion using omni-directional vision and certainty grids

This thesis describes a method to create local maps from an omni-directional vision system (ODVS) mounted on a mobile robot. Range finding is performed by a structure-from-motion method, which recovers the three-dimensional position of objects in the environment from omni-directional images. This leads to map-making, which is accomplished using certainty grids to fuse information from multiple readings into a two-dimensional world model. The system is demonstrated both on noise-free data from a custom-built simulator and on real data from an omni-directional vision system on-board a mobile robot. Finally, to account for the particular error characteristics of a real omni-directional vision sensor, a new sensor model for the certainty grid framework is also created and compared to the traditional sonar sensor model

Practical Euclidean reconstruction of buildings.

Chou Yun-Sum, Bailey.Thesis (M.Phil.)--Chinese University of Hong Kong, 2001.Includes bibliographical references (leaves 89-92).Abstracts in English and Chinese.List of SymbolChapter Chapter 1 --- IntroductionChapter 1.1 --- The Goal: Euclidean Reconstruction --- p.1Chapter 1.2 --- Historical background --- p.2Chapter 1.3 --- Scope of the thesis --- p.2Chapter 1.4 --- Thesis Outline --- p.3Chapter Chapter 2 --- An introduction to stereo vision and 3D shape reconstructionChapter 2.1 --- Homogeneous Coordinates --- p.4Chapter 2.2 --- Camera ModelChapter 2.2.1 --- Pinhole Camera Model --- p.5Chapter 2.3 --- Camera Calibration --- p.11Chapter 2.4 --- Geometry of Binocular System --- p.14Chapter 2.5 --- Stereo Matching --- p.15Chapter 2.5.1 --- Accuracy of Corresponding Point --- p.17Chapter 2.5.2 --- The Stereo Matching Approach --- p.18Chapter 2.5.2.1 --- Intensity-based stereo matching --- p.19Chapter 2.5.2.2 --- Feature-based stereo matching --- p.20Chapter 2.5.3 --- Matching Constraints --- p.20Chapter 2.6 --- 3D Reconstruction --- p.22Chapter 2.7 --- Recent development on self calibration --- p.24Chapter 2.8 --- Summary of the Chapter --- p.25Chapter Chapter 3 --- Camera CalibrationChapter 3.1 --- Introduction --- p.26Chapter 3.2 --- Camera Self-calibration --- p.27Chapter 3.3 --- Self-calibration under general camera motion --- p.27Chapter 3.3.1 --- The absolute Conic Based Techniques --- p.28Chapter 3.3.2 --- A Stratified approach for self-calibration by Pollefeys --- p.33Chapter 3.3.3 --- Pollefeys self-calibration with Absolute Quadric --- p.34Chapter 3.3.4 --- Newsam's self-calibration with linear algorithm --- p.34Chapter 3.4 --- Camera Self-calibration under specially designed motion sequenceChapter 3.4. 1 --- Hartley's self-calibration by pure rotations --- p.35Chapter 3.4.1.1 --- Summary of the AlgorithmChapter 3.4.2 --- Pollefeys self-calibration with variant focal length --- p.36Chapter 3.4.2.1 --- Summary of the AlgorithmChapter 3.4.3 --- Faugeras self-calibration of a 1D Projective Camera --- p.38Chapter 3.5 --- Summary of the Chapter --- p.39Chapter Chapter 4 --- Self-calibration under Planar motionsChapter 4.1 --- Introduction --- p.40Chapter 4.2 --- 1D Projective Camera Self-calibration --- p.41Chapter 4.2.1 --- 1-D camera model --- p.42Chapter 4.2.2 --- 1-D Projective Camera Self-calibration Algorithms --- p.44Chapter 4.2.3 --- Planar motion detection --- p.45Chapter 4.2.4 --- Self-calibration under horizontal planar motions --- p.46Chapter 4.2.5 --- Self-calibration under three different planar motions --- p.47Chapter 4.2.6 --- Result analysis on self-calibration Experiments --- p.49Chapter 4.3 --- Essential Matrix and Triangulation --- p.51Chapter 4.4 --- Merge of Partial 3D models --- p.51Chapter 4.5 --- Summary of the Reconstruction Algorithms --- p.53Chapter 4.6 --- Experimental ResultsChapter 4.6.1 --- Experiment 1 : A Simulated Box --- p.54Chapter 4.6.2 --- Experiment 2 : A Real Building --- p.57Chapter 4.6.3 --- Experiment 3 : A Sun Flower --- p.58Chapter 4.7 --- Conclusion --- p.59Chapter Chapter 5 --- Building Reconstruction using a linear camera self- calibration techniqueChapter 5.1 --- Introduction --- p.60Chapter 5.2 --- Metric Reconstruction from Partially Calibrated imageChapter 5.2.1 --- Partially Calibrated Camera --- p.62Chapter 5.2.2 --- Optimal Computation of Fundamental Matrix (F) --- p.63Chapter 5.2.3 --- Linearly Recovering Two Focal Lengths from F --- p.64Chapter 5.2.4 --- Essential Matrix and Triangulation --- p.66Chapter 5.3 --- Experiments and Discussions --- p.67Chapter 5.4 --- Conclusion --- p.71Chapter Chapter 6 --- Refine the basic model with detail depth information by a Model-Based Stereo techniqueChapter 6.1 --- Introduction --- p.72Chapter 6.2 --- Model Based Epipolar GeometryChapter 6.2.1 --- Overview --- p.74Chapter 6.2.2 --- Warped offset image preparation --- p.76Chapter 6.2.3 --- Epipolar line calculation --- p.78Chapter 6.2.4 --- Actual corresponding point finding by stereo matching --- p.80Chapter 6.2.5 --- Actual 3D point generated by Triangulation --- p.80Chapter 6.3 --- Summary of the Algorithms --- p.81Chapter 6.4 --- Experiments and discussions --- p.83Chapter 6.5 --- Conclusion --- p.85Chapter Chapter 7 --- ConclusionsChapter 7.1 --- Summary --- p.86Chapter 7.2 --- Future Work --- p.88BIBLIOGRAPHY --- p.8

Model-based computer vision: motion analysis, motion-based segmentation, 3D object recognition.

by Man-lee Liu.Thesis (M.Phil.)--Chinese University of Hong Kong, 1998.Includes bibliographical references (leaves 143-151).LIST OF TABLES --- p.viLIST OF FIGURES --- p.xiiCHAPTERChapter 1 --- Introduction --- p.1Chapter 1.1 --- Model-based Motion Analysis --- p.2Chapter 1.1.1 --- With 3D-to-3D Point Correspondences --- p.4Chapter 1.1.2 --- With 2D-to-3D Point Correspondences --- p.5Chapter 1.1.3 --- With 2D-to-2D Point Correspondences --- p.6Chapter 1.2 --- Motion-based Segmentation --- p.7Chapter 1.3 --- 3D Object Recognition --- p.8Chapter 1.4 --- Organization of the Thesis --- p.8Chapter 2 --- Literature Review and Summary of Contributions --- p.10Chapter 2.1 --- Model-based Motion Analysis --- p.10Chapter 2.1.1 --- With 3D-to-3D Point Correspondences --- p.10Chapter 2.1.2 --- With 2D-to-3D Point Correspondences --- p.13Chapter 2.1.2.1 --- An Iterative Approach: Lowe's Algorithm --- p.18Chapter 2.1.2.2 --- A Linear Approach: Faugeras's Algorithm --- p.19Chapter 2.1.3 --- With 2D-to-2D Point Correspondences --- p.22Chapter 2.2 --- Motion-based Segmentation --- p.27Chapter 2.3 --- 3D Object Recognition --- p.28Chapter 2.4 --- Summary of Contributions --- p.30Chapter 3 --- Model-based Motion Analysis with 2D-to-3D Point Correspondences --- p.34Chapter 3.1 --- A new Iterative Algorithm for the Perspective-4-point Problem: TL-algorithm --- p.34Chapter 3.1.1 --- Algorithm --- p.35Chapter 3.1.2 --- Experiment --- p.37Chapter 3.1.2.1 --- Experiment using Synthetic Data --- p.38Chapter 3.1.2.2 --- Experiment using Real Data --- p.42Chapter 3.2 --- An Enhancement of Faugeras's Algorithm --- p.42Chapter 3.2.1 --- Experimental Comparison between the Original Faugeras's Algorithm and the Modified One --- p.44Chapter 3.2.1.1 --- Experiment One: Fixed Motion --- p.44Chapter 3.2.1.2 --- Experiment Two: Using Motion Generated Ran- domly --- p.50Chapter 3.2.2 --- Discussion --- p.54Chapter 3.3 --- A new Linear Algorithm for the Model-based Motion Analysis: Six-point Algorithm --- p.55Chapter 3.3.1 --- General Information of the Six-point Algorithm --- p.55Chapter 3.3.2 --- Original Version of the Six-point Algorithm --- p.56Chapter 3.3.2.1 --- Linear Solution Part --- p.56Chapter 3.3.2.2 --- Constraint Satisfaction --- p.58Use of Representation of Rotations by Quaternion --- p.62Use of Singular Value Decomposition --- p.62Determination of the translational matrix --- p.63Chapter 3.3.3 --- Second Version of the Six-point Algorithm --- p.64Chapter 3.3.4 --- Experiment --- p.65Chapter 3.3.4.1 --- With Synthetic Data --- p.66Experiment One: With Fixed Motion --- p.66Experiment Two: With Motion Generated Randomly --- p.77Chapter 3.3.4.2 --- With Real Data --- p.93Chapter 3.3.5 --- Summary of the Six-Point Algorithm --- p.93Chapter 3.3.6 --- A Visual Tracking System by using Six-point Algorithm --- p.95Chapter 3.4 --- Comparison between TL-algorithm and Six-point Algorithm developed --- p.97Chapter 3.5 --- Summary --- p.102Chapter 4 --- Motion-based Segmentation --- p.104Chapter 4.1 --- A new Approach with 3D-to-3D Point Correspondences --- p.104Chapter 4.1.1 --- Algorithm --- p.105Chapter 4.1.2 --- Experiment --- p.109Chapter 4.2 --- A new Approach with 2D-to-3D Point Correspondences --- p.112Chapter 4.2.1 --- Algorithm --- p.112Chapter 4.2.2 --- Experiment --- p.116Chapter 4.2.2.1 --- Experiment using synthetic data --- p.116Chapter 4.2.2.2 --- Experiment using real image sequence --- p.119Chapter 4.3 --- Summary --- p.119Chapter 5 --- 3D Object Recognition --- p.121Chapter 5.1 --- Proposed Algorithm for the 3D Object Recognition --- p.122Chapter 5.1.1 --- Hypothesis step --- p.122Chapter 5.1.2 --- Verification step --- p.124Chapter 5.2 --- 3D Object Recognition System --- p.125Chapter 5.2.1 --- System in Matlab: --- p.126Chapter 5.2.2 --- System in Visual C++ --- p.129Chapter 5.3 --- Experiment --- p.131Chapter 5.3.1 --- System in Matlab --- p.132Chapter 5.3.2 --- System in Visual C++ --- p.136Chapter 5.4 --- Summary --- p.139Chapter 6 --- Conclusions --- p.140REFERENCES --- p.142APPENDIXChapter A --- Representation of Rotations by Quaternion --- p.152Chapter B --- Constrained Optimization --- p.15

Videos in Context for Telecommunication and Spatial Browsing

The research presented in this thesis explores the use of videos embedded in panoramic imagery to transmit spatial and temporal information describing remote environments and their dynamics. Virtual environments (VEs) through which users can explore remote locations are rapidly emerging as a popular medium of presence and remote collaboration. However, capturing visual representation of locations to be used in VEs is usually a tedious process that requires either manual modelling of environments or the employment of specific hardware. Capturing environment dynamics is not straightforward either, and it is usually performed through specific tracking hardware. Similarly, browsing large unstructured video-collections with available tools is difficult, as the abundance of spatial and temporal information makes them hard to comprehend. At the same time, on a spectrum between 3D VEs and 2D images, panoramas lie in between, as they offer the same 2D images accessibility while preserving 3D virtual environments surrounding representation. For this reason, panoramas are an attractive basis for videoconferencing and browsing tools as they can relate several videos temporally and spatially. This research explores methods to acquire, fuse, render and stream data coming from heterogeneous cameras, with the help of panoramic imagery. Three distinct but interrelated questions are addressed. First, the thesis considers how spatially localised video can be used to increase the spatial information transmitted during video mediated communication, and if this improves quality of communication. Second, the research asks whether videos in panoramic context can be used to convey spatial and temporal information of a remote place and the dynamics within, and if this improves users' performance in tasks that require spatio-temporal thinking. Finally, the thesis considers whether there is an impact of display type on reasoning about events within videos in panoramic context. These research questions were investigated over three experiments, covering scenarios common to computer-supported cooperative work and video browsing. To support the investigation, two distinct video+context systems were developed. The first telecommunication experiment compared our videos in context interface with fully-panoramic video and conventional webcam video conferencing in an object placement scenario. The second experiment investigated the impact of videos in panoramic context on quality of spatio-temporal thinking during localization tasks. To support the experiment, a novel interface to video-collection in panoramic context was developed and compared with common video-browsing tools. The final experimental study investigated the impact of display type on reasoning about events. The study explored three adaptations of our video-collection interface to three display types. The overall conclusion is that videos in panoramic context offer a valid solution to spatio-temporal exploration of remote locations. Our approach presents a richer visual representation in terms of space and time than standard tools, showing that providing panoramic contexts to video collections makes spatio-temporal tasks easier. To this end, videos in context are suitable alternative to more difficult, and often expensive solutions. These findings are beneficial to many applications, including teleconferencing, virtual tourism and remote assistance