Wavelet based stereo images reconstruction using depth images

It is believed by many that three-dimensional (3D) television will be the next logical development toward a more natural and vivid home entertaiment experience. While classical 3D approach requires the transmission of two video streams, one for each view, 3D TV systems based on depth image rendering (DIBR) require a single stream of monoscopic images and a second stream of associated images usually termed depth images or depth maps, that contain per-pixel depth information. Depth map is a two-dimensional function that contains information about distance from camera to a certain point of the object as a function of the image coordinates. By using this depth information and the original image it is possible to reconstruct a virtual image of a nearby viewpoint by projecting the pixels of available image to their locations in 3D space and finding their position in the desired view plane. One of the most significant advantages of the DIBR is that depth maps can be coded more efficiently than two streams corresponding to left and right view of the scene, thereby reducing the bandwidth required for transmission, which makes it possible to reuse existing transmission channels for the transmission of 3D TV. This technique can also be applied for other 3D technologies such as multimedia systems. In this paper we propose an advanced wavelet domain scheme for the reconstruction of stereoscopic images, which solves some of the shortcommings of the existing methods discussed above. We perform the wavelet transform of both the luminance and depth images in order to obtain significant geometric features, which enable more sensible reconstruction of the virtual view. Motion estimation employed in our approach uses Markov random field smoothness prior for regularization of the estimated motion field. The evaluation of the proposed reconstruction method is done on two video sequences which are typically used for comparison of stereo reconstruction algorithms. The results demonstrate advantages of the proposed approach with respect to the state-of-the-art methods, in terms of both objective and subjective performance measures

Geometric smoothing and modeling of 3D graphics

The great challenge in computer graphics and geometric-aided design is to devise computationally efficient and optimal algorithms for estimating 3D models contaminated by noise and preserving their geometrical and topological structure. Motivated by the good performance of anisotropic diffusion in 2D image processing, we propose in this thesis a vertex-based nonlinear flow for 3D mesh smoothing by solving a discrete partial differential equation. The core idea behind our proposed technique is to use geometric insight in helping construct an efficient and fast 3D mesh smoothing strategy to fully preserve the geometric structure of the data. Illustrating experimental results demonstrate a much improved performance of the proposed approach in comparison with existing methods currently used in 3D mesh smoothing. The major part of this thesis is devoted to a joint exploitation of geometry and topology to design new skeletal graph representation of 3D shapes in the Morse-theoretic framework. We present a multiresolution skeletal graph for topological coding of 3D shapes. The proposed skeletonization algorithm encodes a 3D object into a topological graph using a normalized mixture distance function. The approach is accurate, invariant to Euclidean transformations, computationally efficient, and preserves topology. Experimental results demonstrate the potential of the proposed topological graph which may be used as a shape signature for 3D object matching and retrieval, and also for skeletal animatio

DGCM-Net: Dense Geometrical Correspondence Matching Network for Incremental Experience-Based Robotic Grasping.

This article presents a method for grasping novel objects by learning from experience. Successful attempts are remembered and then used to guide future grasps such that more reliable grasping is achieved over time. To transfer the learned experience to unseen objects, we introduce the dense geometric correspondence matching network (DGCM-Net). This applies metric learning to encode objects with similar geometry nearby in feature space. Retrieving relevant experience for an unseen object is thus a nearest neighbor search with the encoded feature maps. DGCM-Net also reconstructs 3D-3D correspondences using the view-dependent normalized object coordinate space to transform grasp configurations from retrieved samples to unseen objects. In comparison to baseline methods, our approach achieves an equivalent grasp success rate. However, the baselines are significantly improved when fusing the knowledge from experience with their grasp proposal strategy. Offline experiments with a grasping dataset highlight the capability to transfer grasps to new instances as well as to improve success rate over time from increasing experience. Lastly, by learning task-relevant grasps, our approach can prioritize grasp configurations that enable the functional use of objects

Efficient Generation of Shape-Based Reference Frames for the Corpus Callosum for DTI-based Connectivity Analysis

Yushkevich et.al. [17, 18] established a PDE-based deformable modeling approach called continuous medial representation (cm-rep), in which the geometric relationship between the medial axis of a 3D object and its boundary is captured. Continuous medial description of an object not only provides useful shape features for object characterization and comparison; it also imposes a shape-based reference frame on the interior of that object. Such a reference frame provides a useful means of representing different instances of an anatomical structure using a common canonical parametrization domain. This paper presents an efficient method to construct continuous medial shape models for 2D objects. A closed form solution for the ordinary differential equation (ODE) is derived via Pythagorean hodograph (PH) curves. That closed form solution reduces the computation complexity from solving an ODE system to pure algebraic manipulation. Using this method, we generate shape-based reference frames, and demonstrate how they can be applied to the analysis of anatomical connectivity of corpora callosa, obtained by fiber tracking in diffusion tensor magnetic resonance imaging (DTI) in a chromosome 22q11.2 deletion syndrome study

Assessment of a photogrammetric approach for urban DSM extraction from tri-stereoscopic satellite imagery

Built-up environments are extremely complex for 3D surface modelling purposes. The main distortions that hamper 3D reconstruction from 2D imagery are image dissimilarities, concealed areas, shadows, height discontinuities and discrepancies between smooth terrain and man-made features. A methodology is proposed to improve automatic photogrammetric extraction of an urban surface model from high resolution satellite imagery with the emphasis on strategies to reduce the effects of the cited distortions and to make image matching more robust. Instead of a standard stereoscopic approach, a digital surface model is derived from tri-stereoscopic satellite imagery. This is based on an extensive multi-image matching strategy that fully benefits from the geometric and radiometric information contained in the three images. The bundled triplet consists of an IKONOS along-track pair and an additional near-nadir IKONOS image. For the tri-stereoscopic study a densely built-up area, extending from the centre of Istanbul to the urban fringe, is selected. The accuracy of the model extracted from the IKONOS triplet, as well as the model extracted from only the along-track stereopair, are assessed by comparison with 3D check points and 3D building vector data