Algorithms for 3D data estimation from single-pixel ToF sensors and stereo vision systems

Depth Map Estimation from stereo devices and time of flight range cameras have been a challenging issues in Computer Vision. Distance Estimations from single-pixel histograms of time of flight sensors are exploited in numerous fields. Beyond the several drawbacks such as degradation caused by strong ambient light, scattered and multi-path possibilities, most of the prediction algorithms could be applied to resolve these problems effectively. As these two different tasks are handled in connection with each other, supervised approaches are considered since they provide more robust results. These results are used to train the model to improve three- dimensional geometry information and against major difficulties such as complicated patterns and objects. These approaches are observed according to their accuracy with help of metrics and get improved their performances. This thesis focuses on the analysis of Time-of-Flight and stereo vision systems for depth map estimation and single-pixel distance prediction. State of art algorithms are compared and implemented with additional strategies which are integrated to minimize the error ratio. The histograms which are obtained from Time of Flight Sensor Simulation are exploited as a dataset for single-pixel distance prediction and after that, NYU Dataset is selected for depth map estimation

MISR-GOES 3D Winds: Implications for Future LEO-GEO and LEO-LEO Winds

Global wind observations are fundamental for studying weather and climate dynamics and for operational forecasting. Most wind measurements come from atmospheric motion vectors (AMVs) by tracking the displacement of cloud or water vapor features. These AMVs generally rely on thermal infrared (IR) techniques for their height assignments, which are subject to large uncertainties in the presence of weak or reversed vertical temperature gradients near the planetary boundary layer (PBL)and tropopause folds. Stereo imaging can overcome the height assignment problem using geometric parallax for feature height determination. In this study we develop a stereo 3D-Wind algorithm to simultaneously retrieve AMV and height from geostationary (GEO) and low Earth orbit (LEO) satellite imagery and apply it to collocated Geostationary Operational Environmental Satellite (GOES)and Multi-angle Imaging SpectroRadiometer (MISR) imagery. The new algorithm improves AMV and height relative to products from GOES or MISR alone, with an estimated accuracy of <0.5 m/s in AMV and <200 m in height with 2.2 km sampling. The algorithm can be generalized to other LEO-GEO or LEO-LEO combinations for greater spatiotemporal coverage. The technique demonstrated with MISR and GOES has important implications for future high-quality AMV observations, for which a low-cost constellation of CubeSats can play a vital role

Synergy of stereo cloud top height and ORAC optimal estimation cloud retrieval: evaluation and application to AATSR

In this paper we evaluate the retrievals of cloud top height when stereo derived heights are combined with the radiometric cloud top heights retrieved from the ORAC (Optimal Retrieval of Aerosol and Cloud) algorithm. This is performed in a mathematically rigorous way using the ORAC optimal estimation retrieval framework, which includes the facility to use independent a priori information. Key to the use of a priori information is a characterisation of their associated uncertainty. This paper demonstrates the improvements that are possible using this approach and also considers their impact on the microphysical cloud parameters retrieved. The AATSR instrument has two views and three thermal channels so is well placed to demonstrate the synergy of the two techniques. The stereo retrieval is able to improve the accuracy of the retrieved cloud top height when compared to collocated Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observations (CALIPSO), particularly in the presence of boundary layer inversions and high clouds. The impact on the microphysical properties of the cloud such as optical depth and effective radius was evaluated and found to be very small with the biggest differences occurring over bright land surfaces and for high clouds. Overall the cost of the retrievals increased indicating a poorer radiative fit of radiances to the cloud model, which currently uses a single layer cloud model. Best results and improved fit to the radiances may be obtained in the future if a multi-layer model is used

Multimodal Stereoscopic Movie Summarization Conforming to Narrative Characteristics

Video summarization is a timely and rapidly developing research field with broad commercial interest, due to the increasing availability of massive video data. Relevant algorithms face the challenge of needing to achieve a careful balance between summary compactness, enjoyability, and content coverage. The specific case of stereoscopic 3D theatrical films has become more important over the past years, but not received corresponding research attention. In this paper, a multi-stage, multimodal summarization process for such stereoscopic movies is proposed, that is able to extract a short, representative video skim conforming to narrative characteristics from a 3D film. At the initial stage, a novel, low-level video frame description method is introduced (frame moments descriptor) that compactly captures informative image statistics from luminance, color, optical flow, and stereoscopic disparity video data, both in a global and in a local scale. Thus, scene texture, illumination, motion, and geometry properties may succinctly be contained within a single frame feature descriptor, which can subsequently be employed as a building block in any key-frame extraction scheme, e.g., for intra-shot frame clustering. The computed key-frames are then used to construct a movie summary in the form of a video skim, which is post-processed in a manner that also considers the audio modality. The next stage of the proposed summarization pipeline essentially performs shot pruning, controlled by a user-provided shot retention parameter, that removes segments from the skim based on the narrative prominence of movie characters in both the visual and the audio modalities. This novel process (multimodal shot pruning) is algebraically modeled as a multimodal matrix column subset selection problem, which is solved using an evolutionary computing approach. Subsequently, disorienting editing effects induced by summarization are dealt with, through manipulation of the video skim. At the last step, the skim is suitably post-processed in order to reduce stereoscopic video defects that may cause visual fatigue