Contour Generator Points for Threshold Selection and a Novel Photo-Consistency Measure for Space Carving

Space carving has emerged as a powerful method for multiview scene reconstruction. Although a wide variety of methods have been proposed, the quality of the reconstruction remains highly-dependent on the photometric consistency measure, and the threshold used to carve away voxels. In this paper, we present a novel photo-consistency measure that is motivated by a multiset variant of the chamfer distance. The new measure is robust to high amounts of within-view color variance and also takes into account the projection angles of back-projected pixels. Another critical issue in space carving is the selection of the photo-consistency threshold used to determine what surface voxels are kept or carved away. In this paper, a reliable threshold selection technique is proposed that examines the photo-consistency values at contour generator points. Contour generators are points that lie on both the surface of the object and the visual hull. To determine the threshold, a percentile ranking of the photo-consistency values of these generator points is used. This improved technique is applicable to a wide variety of photo-consistency measures, including the new measure presented in this paper. Also presented in this paper is a method to choose between photo-consistency measures, and voxel array resolutions prior to carving using receiver operating characteristic (ROC) curves

Fast feature matching for detailed point cloud generation

Structure from motion is a very popular technique for obtaining three-dimensional point cloud-based reconstructions of objects from un-organised sets of images by analysing the correspondences between feature points detected in those images. However, the point clouds stemming from usual feature point extractors such as SIFT are frequently too sparse for reliable surface recovery. In this paper we show that alternate feature descriptors such as A-KAZE, which provide denser coverage of images, yield better results and more detailed point clouds. Unfortunately, the use of a dramatically increased number of points per image poses a computational challenge. We propose a technique based on epipolar geometry restrictions to significantly cut down on processing time and an efficient implementation thereof on a GPU

Realtime Color Stereovision Processing

Recent developments in aviation have made micro air vehicles (MAVs) a reality. These featherweight palm-sized radio-controlled flying saucers embody the future of air-to-ground combat. No one has ever successfully implemented an autonomous control system for MAVs. Because MAVs are physically small with limited energy supplies, video signals offer superiority over radar for navigational applications. This research takes a step forward in real time machine vision processing. It investigates techniques for implementing a real time stereovision processing system using two miniature color cameras. The effects of poor-quality optics are overcome by a robust algorithm, which operates in real time and achieves frame rates up to 10 fps in ideal conditions. The vision system implements innovative work in the following five areas of vision processing: fast image registration preprocessing, object detection, feature correspondence, distortion-compensated ranging, and multi scale nominal frequency-based object recognition. Results indicate that the system can provide adequate obstacle avoidance feedback for autonomous vehicle control. However, typical relative position errors are about 10%-to high for surveillance applications. The range of operation is also limited to between 6 - 30 m. The root of this limitation is imprecise feature correspondence: with perfect feature correspondence the range would extend to between 0.5 - 30 m. Stereo camera separation limits the near range, while optical resolution limits the far range. Image frame sizes are 160x120 pixels. Increasing this size will improve far range characteristics but will also decrease frame rate. Image preprocessing proved to be less appropriate than precision camera alignment in this application. A proof of concept for object recognition shows promise for applications with more precise object detection. Future recommendations are offered in all five areas of vision processing

Development and evaluation of a multiscale keypoint detector based on complex wavelets

This thesis develops a multiscale keypoint detector and descriptor based on the Dual-Tree Complex Wavelet Transform (DTCWT). First, we develop a scale-space framework called the 4S-DTCWT that uses the dyadic decomposition of the DTCWT but achieves denser sampling in scale by interleaving several DTCWT trees, leading to reduced scale-related aliasing. This forms the foundation for the rest of our work. Then, we present a new DTCWT based keypoint detector (BTK), which exhibits improved spatial localisation owing to the use of a more selective cornerness measure and keypoint localisation in individual levels in the 4S-DTCWT. A number of scale refinement approaches are investigated. The improved keypoint position and scale localisation directly leads to more robust image characterisation using DTCWT based visual descriptors. We also present some ways of speeding up both the descriptor and the matching computations. These changes make it possible to use the system in practical scenarios. We develop a novel, fully automated framework for the evaluation of keypoint detectors and descriptors. This includes a new dataset containing 3978 calibrated images from 2 cameras of 39 different toy cars on a turntable. The dataset, calibration images, inter-camera calibration, rotational calibration and test scripts are publicly available. We establish ground truth correspondences using a three-image setup, with fixed angular separation between two of the three views, thus reducing the dependency on angular separation when compared to conventional epipolar line search. Various keypoint detectors and descriptors were compared with DTCWT based methods using this framework. To the extent possible, we separated the evaluation of the keypoint detectors from that of the descriptors. The main conclusions were that DTCWT based methods can achieve a performance comparable, if not superior, to that of established methods. We also showed that, although repeatability of keypoint detections falls off reasonably steeply with change in viewing angle, conditioned on an associated keypoint being detected at a reasonably correct corresponding location, descriptor similarity is hardly affected by viewpoint variation. Finally, we show how an evaluation that is based purely on the prior knowledge of the geometry of the scene can be useful in eliminating the inaccuracies involved in appearance based evaluations. This uses an enhanced epipolar constraint that exploits both positions and scales of keypoints to constrain the range of possible matches

Nonlinear force-free coronal magnetic stereoscopy

Getting insights into the 3D structure of the solar coronal magnetic field have been done in the past by two completely different approaches: (1.) Nonlinear force-free field (NLFFF) extrapolations, which use photospheric vector magnetograms as boundary condition. (2.) Stereoscopy of coronal magnetic loops observed in EUV coronal images from different vantage points. Both approaches have their strength and weaknesses. Extrapolation methods are sensitive to noise and inconsistencies in the boundary data and the accuracy of stereoscopy is affected by the ability of identifying the same structure in different images and by the separation angle between the view directions. As a consequence, for the same observational data, the computed 3D coronal magnetic field with the two methods do not necessarily coincide. In an earlier work (Paper I) we extended our NLFFF optimization code by the inclusion of stereoscopic constrains. The method was successfully tested with synthetic data and within this work we apply the newly developed code to a combined data-set from SDO/HMI, SDO/AIA and the two STEREO spacecraft. The extended method (called S-NLFFF) contains an additional term that monitors and minimizes the angle between the local magnetic field direction and the orientation of the 3D coronal loops reconstructed by stereoscopy. We find that prescribing the shape of the 3D stereoscopically reconstructed loops the S-NLFFF method leads to a much better agreement between the modeled field and the stereoscopically reconstructed loops. We also find an appreciable decrease by a factor of two in the angle between the current and the magnetic field which indicates the improved quality of the force-free solution obtained by S-NLFFF.Comment: 9 pages, 7 figure

Contribution towards a fast stereo dense matching.

Stereo matching is important in the area of computer vision as it is the basis of the reconstruction process. Many applications require 3D reconstruction such as view synthesis, robotics... The main task of matching uncalibrated images is to determine the corresponding pixels and other features where the motion between these images and the camera parameters is unknown. Although some methods have been carried out over the past two decades on the matching problem, most of these methods are not practical and difficult to implement. Our approach considers a reliable image edge features in order to develop a fast and practical method. Therefore, we propose a fast stereo matching algorithm combining two different approaches for matching as the image is segmented into two sets of regions: edge regions and non-edge regions. We have used an algebraic method that preserves disparity continuity at the object continuous surfaces. Our results demonstrate that we gain a speed dense matching while the implementation is kept simple and straightforward.Dept. of Computer Science. Paper copy at Leddy Library: Theses & Major Papers - Basement, West Bldg. / Call Number: Thesis2005 .Z42. Source: Masters Abstracts International, Volume: 44-03, page: 1420. Thesis (M.Sc.)--University of Windsor (Canada), 2005

Local Feature Selection and Global Energy Optimization in Stereo

The human brain can fuse two slightly different views from left and right eyes and perceive depth. This process of stereopsis entails identifying matching locations in the two images and recovering the depth from their disparity. This can be done only approximately: ambiguity arising from such factors as noise, periodicity, and large regions of constan