Layered Interpretation of Street View Images

We propose a layered street view model to encode both depth and semantic information on street view images for autonomous driving. Recently, stixels, stix-mantics, and tiered scene labeling methods have been proposed to model street view images. We propose a 4-layer street view model, a compact representation over the recently proposed stix-mantics model. Our layers encode semantic classes like ground, pedestrians, vehicles, buildings, and sky in addition to the depths. The only input to our algorithm is a pair of stereo images. We use a deep neural network to extract the appearance features for semantic classes. We use a simple and an efficient inference algorithm to jointly estimate both semantic classes and layered depth values. Our method outperforms other competing approaches in Daimler urban scene segmentation dataset. Our algorithm is massively parallelizable, allowing a GPU implementation with a processing speed about 9 fps.Comment: The paper will be presented in the 2015 Robotics: Science and Systems Conference (RSS

Wavelet based stereo images reconstruction using depth images

It is believed by many that three-dimensional (3D) television will be the next logical development toward a more natural and vivid home entertaiment experience. While classical 3D approach requires the transmission of two video streams, one for each view, 3D TV systems based on depth image rendering (DIBR) require a single stream of monoscopic images and a second stream of associated images usually termed depth images or depth maps, that contain per-pixel depth information. Depth map is a two-dimensional function that contains information about distance from camera to a certain point of the object as a function of the image coordinates. By using this depth information and the original image it is possible to reconstruct a virtual image of a nearby viewpoint by projecting the pixels of available image to their locations in 3D space and finding their position in the desired view plane. One of the most significant advantages of the DIBR is that depth maps can be coded more efficiently than two streams corresponding to left and right view of the scene, thereby reducing the bandwidth required for transmission, which makes it possible to reuse existing transmission channels for the transmission of 3D TV. This technique can also be applied for other 3D technologies such as multimedia systems. In this paper we propose an advanced wavelet domain scheme for the reconstruction of stereoscopic images, which solves some of the shortcommings of the existing methods discussed above. We perform the wavelet transform of both the luminance and depth images in order to obtain significant geometric features, which enable more sensible reconstruction of the virtual view. Motion estimation employed in our approach uses Markov random field smoothness prior for regularization of the estimated motion field. The evaluation of the proposed reconstruction method is done on two video sequences which are typically used for comparison of stereo reconstruction algorithms. The results demonstrate advantages of the proposed approach with respect to the state-of-the-art methods, in terms of both objective and subjective performance measures

Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications

Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications

Cosmic Ray in the Northern Hemisphere: Results from the Telescope Array Experiment

The Telescope Array (TA) is the largest ultrahigh energy (UHE) cosmic ray observatory in the northern hemisphere TA is a hybrid experiment with a unique combination of fluorescence detectors and a stand-alone surface array of scintillation counters. We will present the spectrum measured by the surface array alone, along with those measured by the fluorescence detectors in monocular, hybrid, and stereo mode. The composition results from stereo TA data will be discussed. Our report will also include results from the search for correlations between the pointing directions of cosmic rays, seen by the TA surface array, with active galactic nuclei.Comment: 8 pages 11 figure, Proceedings of the APS Division of Particle and Fields (DPF) Meeting, Aug 2011, Brown University, Providence, RI, US

Fluids mobilization in Arabia Terra, Mars: depth of pressurized reservoir from mounds self-similar clustering

Arabia Terra is a region of Mars where signs of past-water occurrence are recorded in several landforms. Broad and local scale geomorphological, compositional and hydrological analyses point towards pervasive fluid circulation through time. In this work we focus on mound fields located in the interior of three casters larger than 40 km (Firsoff, Kotido and unnamed crater 20 km to the east) and showing strong morphological and textural resemblance to terrestrial mud volcanoes and spring-related features. We infer that these landforms likely testify the presence of a pressurized fluid reservoir at depth and past fluid upwelling. We have performed morphometric analyses to characterize the mound morphologies and consequently retrieve an accurate automated mapping of the mounds within the craters for spatial distribution and fractal clustering analysis. The outcome of the fractal clustering yields information about the possible extent of the percolating fracture network at depth below the craters. We have been able to constrain the depth of the pressurized fluid reservoir between ~2.5 and 3.2 km of depth and hence, we propose that mounds and mounds alignments are most likely associated to the presence of fissure ridges and fluid outflow. Their process of formation is genetically linked to the formation of large intra-crater bulges previously interpreted as large scale spring deposits. The overburden removal caused by the impact crater formation is the inferred triggering mechanism for fluid pressurization and upwelling, that through time led to the formation of the intra-crater bulges and, after compaction and sealing, to the widespread mound fields in their surroundings