Perceptual coding for 3D reconstruction

A primary issue in 3D reconstruction is the realtime efficacy of different coding methods for the multiple decisions among competing 3D solutions. A common model framework making such coding decisions is the boundary limited drift-diffusion model, which has been developed in parallel in various branches of science from quantum physics to economics. A common property of all such models is the linear increase in variance of the diffusion processes over time, implying an inability to focus on the current information in the environment, and the inevitability of a forced random decision in the absence of any reliable evidence. We have developed an alternative, more plausible model framework for Bayesian information accumulation that solves both problems and provides an accurate account of many features of context effects in human 3D reconstruction performance. © 2011 IEEE

DCT and DST based Image Compression for 3D Reconstruction

This paper introduces a new method for 2D image compression whose quality is demonstrated through accurate 3D reconstruction using structured light techniques and 3D reconstruction from multiple viewpoints. The method is based on two discrete transforms: 1) A one-dimensional Discrete Cosine Transform (DCT) is applied to each row of the image. 2) The output from the previous step is transformed again by a one-dimensional Discrete Sine Transform (DST), which is applied to each column of data generating new sets of high-frequency components followed by quantization of the higher frequencies. The output is then divided into two parts where the low-frequency components are compressed by arithmetic coding and the high frequency ones by an efficient minimization encoding algorithm. At decompression stage, a binary search algorithm is used to recover the original high frequency components. The technique is demonstrated by compressing 2D images up to 99% compression ratio. The decompressed images, which include images with structured light patterns for 3D reconstruction and from multiple viewpoints, are of high perceptual quality yielding accurate 3D reconstruction. Perceptual assessment and objective quality of compression are compared with JPEG and JPEG2000 through 2D and 3D RMSE. Results show that the proposed compression method is superior to both JPEG and JPEG2000 concerning 3D reconstruction, and with equivalent perceptual quality to JPEG2000

Coding of details in very low bit-rate video systems

In this paper, the importance of including small image features at the initial levels of a progressive second generation video coding scheme is presented. It is shown that a number of meaningful small features called details should be coded, even at very low data bit-rates, in order to match their perceptual significance to the human visual system. We propose a method for extracting, perceptually selecting and coding of visual details in a video sequence using morphological techniques. Its application in the framework of a multiresolution segmentation-based coding algorithm yields better results than pure segmentation techniques at higher compression ratios, if the selection step fits some main subjective requirements. Details are extracted and coded separately from the region structure and included in the reconstructed images in a later stage. The bet of considering the local background of a given detail for its perceptual selection breaks the concept ofPeer ReviewedPostprint (published version

Multiple image view synthesis for free viewpoint video applications

Interactive audio-visual (AV) applications such as free viewpoint video (FVV) aim to enable unrestricted spatio-temporal navigation within multiple camera environments. Current virtual viewpoint view synthesis solutions for FVV are either purely image-based implying large information redundancy; or involve reconstructing complex 3D models of the scene. In this paper we present a new multiple image view synthesis algorithm that only requires camera parameters and disparity maps. The multi-view synthesis (MVS) approach can be used in any multi-camera environment and is scalable as virtual views can be created given 1 to N of the available video inputs, providing a means to gracefully handle scenarios where camera inputs decrease or increase over time. The algorithm identifies and selects only the best quality surface areas from available reference images, thereby reducing perceptual errors in virtual view reconstruction. Experimental results are presented and verified using both objective (PSNR) and subjective comparisons

Scalable virtual viewpoint image synthesis for multiple camera environments

One of the main aims of emerging audio-visual (AV) applications is to provide interactive navigation within a captured event or scene. This paper presents a view synthesis algorithm that provides a scalable and flexible approach to virtual viewpoint synthesis in multiple camera environments. The multi-view synthesis (MVS) process consists of four different phases that are described in detail: surface identification, surface selection, surface boundary blending and surface reconstruction. MVS view synthesis identifies and selects only the best quality surface areas from the set of available reference images, thereby reducing perceptual errors in virtual view reconstruction. The approach is camera setup independent and scalable as virtual views can be created given 1 to N of the available video inputs. Thus, MVS provides interactive AV applications with a means to handle scenarios where camera inputs increase or decrease over time

High-Performance Embedded Morphological Wavelet Coding

Morphological analysis can be applied in wavelet domain to analyze and represent the position of significant coefficients. New operators have to be introduced which are able to exploit both the multiresolution and the filter bank peculiarities of the subband representation of visual information. In this paper an efficient morphological wavelet coder is proposed. The clustering trend of significant coefficients is captured by a new kind of multi resolution binary dilation operator. The layered and adaptive nature of this subband dilation makes it possible for the coding technique to produce an embedded bit-stream with a modest computational cost and state-of-the-art Rate-Distortion performance. Morphological wavelet coding appears promising because the localized analysis of wavelet coefficient clusters is adequate to capture intrinsic patterns of the source which can have substantial benefits for perceptual or even object-based reconstruction quality concerns. Here we test the performance of our algorithm and compare the effects of different wavelet filters. We obtain state of the art coding performance and good perceptual results both for 2D and 3D images, with a new technique that seems to be well suited for further developments