A data-fusion approach to motion-stereo

This paper introduces a novel method for performing motion--stereo, based on dynamic integration of depth (or its proxy) measures obtained by pairwise stereo matching of video frames. The focus is on the data fusion issue raised by the motion--stereo approach, which is solved within a Kalman filtering framework. Integration occurs along the temporal and spatial dimension, so that the final measure for a pixel results from the combination of measures of the same pixel in time and whose of its neighbors. The method has been validated on both synthetic and natural images, using the simplest stereo matching strategy and a range of different confidence measures, and has been compared to baseline and optimal strategies

NOVEL DENSE STEREO ALGORITHMS FOR HIGH-QUALITY DEPTH ESTIMATION FROM IMAGES

This dissertation addresses the problem of inferring scene depth information from a collection of calibrated images taken from different viewpoints via stereo matching. Although it has been heavily investigated for decades, depth from stereo remains a long-standing challenge and popular research topic for several reasons. First of all, in order to be of practical use for many real-time applications such as autonomous driving, accurate depth estimation in real-time is of great importance and one of the core challenges in stereo. Second, for applications such as 3D reconstruction and view synthesis, high-quality depth estimation is crucial to achieve photo realistic results. However, due to the matching ambiguities, accurate dense depth estimates are difficult to achieve. Last but not least, most stereo algorithms rely on identification of corresponding points among images and only work effectively when scenes are Lambertian. For non-Lambertian surfaces, the brightness constancy assumption is no longer valid. This dissertation contributes three novel stereo algorithms that are motivated by the specific requirements and limitations imposed by different applications. In addressing high speed depth estimation from images, we present a stereo algorithm that achieves high quality results while maintaining real-time performance. We introduce an adaptive aggregation step in a dynamic-programming framework. Matching costs are aggregated in the vertical direction using a computationally expensive weighting scheme based on color and distance proximity. We utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this process over two orders of magnitude. In addressing high accuracy depth estimation, we present a stereo model that makes use of constraints from points with known depths - the Ground Control Points (GCPs) as referred to in stereo literature. Our formulation explicitly models the influences of GCPs in a Markov Random Field. A novel regularization prior is naturally integrated into a global inference framework in a principled way using the Bayes rule. Our probabilistic framework allows GCPs to be obtained from various modalities and provides a natural way to integrate information from various sensors. In addressing non-Lambertian reflectance, we introduce a new invariant for stereo correspondence which allows completely arbitrary scene reflectance (bidirectional reflectance distribution functions - BRDFs). This invariant can be used to formulate a rank constraint on stereo matching when the scene is observed by several lighting configurations in which only the lighting intensity varies

Real-Time Mapping Using Stereoscopic Vision Optimization

This research focuses on efficient methods of generating 2D maps from stereo vision in real-time. Instead of attempting to locate edges between objects, we make the assumption that the representative surfaces of objects in a view provide enough information to generate a map while taking less time to locate during processing. Since all real-time vision processing endeavors are extremely computationally intensive, numerous optimization techniques are applied to allow for a real-time application: horizontal spike smoothing for post-disparity noise, masks to focus on close-proximity objects, melding for object synthesis, and rectangular fitting for object extraction under a planar assumption. Additionally, traditional image transformation mechanisms such as rotation, translation, and scaling are integrated. Results from our research are an encouraging 10Hz with no vision post processing and accuracy up to 11 feet. Finally, vision mapping results are compared to simultaneously collected sonar data in three unique experimental settings

Extraction of spatial information from sterioscopic SAR images

Synthetic Aperture Radar (SAR) is now widely used for generating Digital Elevation Models (DEMs) and has advantages over optical data in terms of availability as it allows all-day and all-weather operations. The stereoscopic SAR method, which allows direct extraction of spatial information in three-dimensional space, has been established for decades. However, the traditional stereoscopic methods developed for SAR data depend on many human operations and need ground control points (GCPs), to set up geometric models. The aims of the thesis are not only to propose a refined rigorous stereoscopic SAR method and a new error model to predict theoretic errors, but also to achieve a higher level of automation and accuracy. By using a weighting matrix, which is derived by considering different observations in the space intersection algorithm, the minimal number of the GCPs required for the refined algorithm is only two. To achieve a high degree of automation, an optimized strategy of parameter selection for the pyramidal image correlation scheme employing a region-growing technique has been proposed. This avoids a trial-and-error approach to produce digital parallax data from the same-side SAR image pairs. A new method to derive GCPs automatically has been developed using a SAR image simulation technique, under the condition that a known DEM chip is available, to minimize human interventions and operator error. The proposed method for providing GCPs and the DEMs generated from space intersection have been incorporated into the procedures for geocoding SAR images to validate the proposed algorithms. The results derived show that the stereoscopic SAR data can be applied to geometric rectification in flat-to-moderate areas, and other applications of extraction of spatial information are promising

Digital Surface Modelling in Developing Countries Using Spaceborne SAR Techniques

Topographic databases at the national level, in the form of Digital Surface Models (DSMs), are required for a large number of applications which have been spurred on by the increased use of Geographic Information Systems (GIS). Ground-Based (surveying, GPS, etc.) and traditional airborne approaches to generating topographic information are proving to be time consuming and costly for applications in developing countries. Where these countries are located in the tropical zone, they are affected by the additional problem of cloud cover which could cause delays for almost 75% of the year in obtaining optical imagery. The Caribbean happens to be one such affected territory that is in need of national digital topographic information for its GIS database developments, 3D visualization of landscapes and for use in the digital ortho-rectification of satellite imagery. The use of Synthetic Aperture Radar (SAR), with its cloud penetrating and day/night imaging capabilities, is emerging as a possible remote sensing tool for use in cloud affected territories. There has been success with airborne single-pass dual antennae systems (e.g. STAR 3i) and the Shuttle Radar Topographic Mapping (SRTM) mission. However, the use of these systems in the Caribbean are restrictive and datasets will not be generally available. The launching of imaging radar satellites such as ERS-1, ERS-2, Radarsat-1 and more recently Envisat have provided additional opportunities for augmenting the technologies available for generating medium accuracy, low cost, topographic information for developing countries by using the techniques of Radargrammetry (StereoSAR) and Interferometric SAR (InSAR). The primary aim of this research was to develop, from scratch, a prototype StereoSAR system based on automatic stereo matching and space intersection algorithms to generate medium accuracy, low cost DSMs, using various influencing parameters without any recourse to ground control points. The result was to be a software package to undertake this process for implementation on a personal computer. The DSMs generated from Radarsat-1 and Envisat SAR imagery were compared with a reference surface from airborne InSAR and conclusions with respect to the quality of the StereoSAR DSMs are presented. Work required to further improve the StereoSAR system is also suggested

Digital Surface Modelling in Developing Countries Using Spaceborne SAR Techniques

Topographic databases at the national level, in the form of Digital Surface Models (DSMs), are required for a large number of applications which have been spurred on by the increased use of Geographic Information Systems (GIS). Ground-Based (surveying, GPS, etc.) and traditional airborne approaches to generating topographic information are proving to be time consuming and costly for applications in developing countries. Where these countries are located in the tropical zone, they are affected by the additional problem of cloud cover which could cause delays for almost 75% of the year in obtaining optical imagery. The Caribbean happens to be one such affected territory that is in need of national digital topographic information for its GIS database developments, 3D visualization of landscapes and for use in the digital ortho-rectification of satellite imagery. The use of Synthetic Aperture Radar (SAR), with its cloud penetrating and day/night imaging capabilities, is emerging as a possible remote sensing tool for use in cloud affected territories. There has been success with airborne single-pass dual antennae systems (e.g. STAR 3i) and the Shuttle Radar Topographic Mapping (SRTM) mission. However, the use of these systems in the Caribbean are restrictive and datasets will not be generally available. The launching of imaging radar satellites such as ERS-1, ERS-2, Radarsat-1 and more recently Envisat have provided additional opportunities for augmenting the technologies available for generating medium accuracy, low cost, topographic information for developing countries by using the techniques of Radargrammetry (StereoSAR) and Interferometric SAR (InSAR). The primary aim of this research was to develop, from scratch, a prototype StereoSAR system based on automatic stereo matching and space intersection algorithms to generate medium accuracy, low cost DSMs, using various influencing parameters without any recourse to ground control points. The result was to be a software package to undertake this process for implementation on a personal computer. The DSMs generated from Radarsat-1 and Envisat SAR imagery were compared with a reference surface from airborne InSAR and conclusions with respect to the quality of the StereoSAR DSMs are presented. Work required to further improve the StereoSAR system is also suggested

Investigation of developments in interferometric synthetic aperture radar until 1994

Bibliography: p. 149-155.This thesis examines the topic of Synthetic Aperture Radar Interferometry in a historical perspective, tracing its development from its beginnings in the 1960s up until May 1994. Applications are listed and airborne and spaceborne implementations reviewed. The underlying theory of interferometry is explained, including a discussion of error sources, and a simulation for point targets is documented to illustrate the interferometric processing steps. The application of the SASAR VHF SAR system to interferometric operation is examined analytically

Foveation for 3D visualization and stereo imaging

Even though computer vision and digital photogrammetry share a number of goals, techniques, and methods, the potential for cooperation between these fields is not fully exploited. In attempt to help bridging the two, this work brings a well-known computer vision and image processing technique called foveation and introduces it to photogrammetry, creating a hybrid application. The results may be beneficial for both fields, plus the general stereo imaging community, and virtual reality applications. Foveation is a biologically motivated image compression method that is often used for transmitting videos and images over networks. It is possible to view foveation as an area of interest management method as well as a compression technique. While the most common foveation applications are in 2D there are a number of binocular approaches as well. For this research, the current state of the art in the literature on level of detail, human visual system, stereoscopic perception, stereoscopic displays, 2D and 3D foveation, and digital photogrammetry were reviewed. After the review, a stereo-foveation model was constructed and an implementation was realized to demonstrate a proof of concept. The conceptual approach is treated as generic, while the implementation was conducted under certain limitations, which are documented in the relevant context. A stand-alone program called Foveaglyph is created in the implementation process. Foveaglyph takes a stereo pair as input and uses an image matching algorithm to find the parallax values. It then calculates the 3D coordinates for each pixel from the geometric relationships between the object and the camera configuration or via a parallax function. Once 3D coordinates are obtained, a 3D image pyramid is created. Then, using a distance dependent level of detail function, spherical volume rings with varying resolutions throughout the 3D space are created. The user determines the area of interest. The result of the application is a user controlled, highly compressed non-uniform 3D anaglyph image. 2D foveation is also provided as an option. This type of development in a photogrammetric visualization unit is beneficial for system performance. The research is particularly relevant for large displays and head mounted displays. Although, the implementation, because it is done for a single user, would possibly be best suited to a head mounted display (HMD) application. The resulting stereo-foveated image can be loaded moderately faster than the uniform original. Therefore, the program can potentially be adapted to an active vision system and manage the scene as the user glances around, given that an eye tracker determines where exactly the eyes accommodate. This exploration may also be extended to robotics and other robot vision applications. Additionally, it can also be used for attention management and the viewer can be directed to the object(s) of interest the demonstrator would like to present (e.g. in 3D cinema). Based on the literature, we also believe this approach should help resolve several problems associated with stereoscopic displays such as the accommodation convergence problem and diplopia. While the available literature provides some empirical evidence to support the usability and benefits of stereo foveation, further tests are needed. User surveys related to the human factors in using stereo foveated images, such as its possible contribution to prevent user discomfort and virtual simulator sickness (VSS) in virtual environments, are left as future work.reviewe