Light field coding with field of view scalability and exemplar-based inter-layer prediction

Light field imaging based on microlens arrays—a.k.a. holoscopic, plenoptic, and integral imaging—has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representations and coding solutions that support arising new manipulation and interaction functionalities. In this context, this paper proposes a novel scalable coding solution that supports a new type of scalability, referred to as field-of-view scalability. The proposed scalable coding solution comprises a base layer compliant with the High Efficiency Video Coding (HEVC) standard, complemented by one or more enhancement layers that progressively allow richer versions of the same light field content in terms of content manipulation and interaction possibilities. In addition, to achieve high-compression performance in the enhancement layers, novel exemplar-based interlayer coding tools are also proposed, namely: 1) a direct prediction based on exemplar texture samples from lower layers and 2) an interlayer compensated prediction using a reference picture that is built relying on an exemplar-based algorithm for texture synthesis. Experimental results demonstrate the advantages of the proposed scalable coding solution to cater to users with different preferences/requirements in terms of interaction functionalities, while providing better rate- distortion performance (independently of the optical setup used for acquisition) compared to HEVC and other scalable light field coding solutions in the literature.info:eu-repo/semantics/acceptedVersio

Scalable Coding of Video Objects

This paper provides a methodology to encode video objects in a scalable manner with regard to both content and quality. Content scalability and quality scalability have been identified as required features in order to support video coding across different environments. Following the object-based approach to coding video, we extend our previous work on motion-based segmentation by using a time recursive approach to segmenting image sequences and decomposing a video "shot" into its constituent objects. Our formulation of the segmentation problem enables us to design a codec in which the information (shape, texture and motion) pertaining to each video object is encoded independently of the other. The multiresolution wavelet decomposition used in encoding texture information is shown to be helpful in providing spatial scalability. Our codec design is also shown to be temporally scalable. This report was accepted for oral presentation at the IEEE International Symposium on Circuits & Systems, Monterey, Calif., May-June 1998

Layer Selection in Progressive Transmission of Motion-Compensated JPEG2000 Video

MCJ2K (Motion-Compensated JPEG2000) is a video codec based on MCTF (Motion- Compensated Temporal Filtering) and J2K (JPEG2000). MCTF analyzes a sequence of images, generating a collection of temporal sub-bands, which are compressed with J2K. The R/D (Rate-Distortion) performance in MCJ2K is better than the MJ2K (Motion JPEG2000) extension, especially if there is a high level of temporal redundancy. MCJ2K codestreams can be served by standard JPIP (J2K Interactive Protocol) servers, thanks to the use of only J2K standard file formats. In bandwidth-constrained scenarios, an important issue in MCJ2K is determining the amount of data of each temporal sub-band that must be transmitted to maximize the quality of the reconstructions at the client side. To solve this problem, we have proposed two rate-allocation algorithms which provide reconstructions that are progressive in quality. The first, OSLA (Optimized Sub-band Layers Allocation), determines the best progression of quality layers, but is computationally expensive. The second, ESLA (Estimated-Slope sub-band Layers Allocation), is sub-optimal in most cases, but much faster and more convenient for real-time streaming scenarios. An experimental comparison shows that even when a straightforward motion compensation scheme is used, the R/D performance of MCJ2K competitive is compared not only to MJ2K, but also with respect to other standard scalable video codecs

Multiple description video coding for stereoscopic 3D

In this paper, we propose an MDC schemes for stereoscopic 3D video. In the literature, MDC has previously been applied in 2D video but not so much in 3D video. The proposed algorithm enhances the error resilience of the 3D video using the combination of even and odd frame based MDC while retaining good temporal prediction efficiency for video over error-prone networks. Improvements are made to the original even and odd frame MDC scheme by adding a controllable amount of side information to improve frame interpolation at the decoder. The side information is also sent according to the video sequence motion for further improvement. The performance of the proposed algorithms is evaluated in error free and error prone environments especially for wireless channels. Simulation results show improved performance using the proposed MDC at high error rates compared to the single description coding (SDC) and the original even and odd frame MDC

The aceToolbox: low-level audiovisual feature extraction for retrieval and classification

In this paper we present an overview of a software platform that has been developed within the aceMedia project, termed the aceToolbox, that provides global and local lowlevel feature extraction from audio-visual content. The toolbox is based on the MPEG-7 eXperimental Model (XM), with extensions to provide descriptor extraction from arbitrarily shaped image segments, thereby supporting local descriptors reflecting real image content. We describe the architecture of the toolbox as well as providing an overview of the descriptors supported to date. We also briefly describe the segmentation algorithm provided. We then demonstrate the usefulness of the toolbox in the context of two different content processing scenarios: similarity-based retrieval in large collections and scene-level classification of still images

Complexity Reduction and Fast Algorithm for 2-D Integer Discrete Wavelet Transform Using Symmetric Mask-Based Scheme

[[abstract]]Wavelet coding has been shown to be better than discrete cosine transform (DCT) in image/video processing. Moreover, it has the feature of scalability, which is involved in modern video standards. This work presents novel algorithms, namely 2-D symmetric mask-based discrete wavelet transform (SMDWT), to improve the critical issue of the 2-D lifting-based discrete wavelet transform (LDWT), and then obtains the benefit of low latency, high-speed operation, and low temporal memory. The SMDWT also has the advantages of high-performance embedded periodic extension boundary treatment, reduced complexity, regular signal coding, short critical path, reduced latency time, and independent subband coding processing. Moreover, the 2-D lifting-based DWT performance can also be easily improved by exploiting appropriate parallel method inherently in SMDWT. Comparing with the normal 2-D 5/3 integer lifting-based DWT the proposed method significantly improves lifting-based latency and complexity in 2-D DWT without degradation in image quality. The algorithm can be applied to real-time image/video applications, such as JPEG2000, MPEG-4 still texture object decoding, and wavelet-based Scalable Video Coding (SVC).[[sponsorship]]IEEE Computer Society, U.S.A.[[notice]]需補會議地點[[conferencetype]]國際[[conferencedate]]20071210~2007121

A proposal for dependent optimization in scalabale region-based coding systems

We address in this paper the problem of optimal coding in the framework of region-based video coding systems, with a special stress on content-based functionalities. We present a coding system that can provide scaled layers (using PSNR or temporal content-based scalability) such that each one has an optimal partition with optimal bit allocation among the resulting regions. This coding system is based on a dependent optimization algorithm that can provide joint optimality for a group of layers or a group of frames.Peer ReviewedPostprint (published version