40 research outputs found
Convex Optimization Based Bit Allocation for Light Field Compression under Weighting and Consistency Constraints
Compared with conventional image and video, light field images introduce the
weight channel, as well as the visual consistency of rendered view, information
that has to be taken into account when compressing the pseudo-temporal-sequence
(PTS) created from light field images. In this paper, we propose a novel frame
level bit allocation framework for PTS coding. A joint model that measures
weighted distortion and visual consistency, combined with an iterative encoding
system, yields the optimal bit allocation for each frame by solving a convex
optimization problem. Experimental results show that the proposed framework is
effective in producing desired distortion distribution based on weights, and
achieves up to 24.7% BD-rate reduction comparing to the default rate control
algorithm.Comment: published in IEEE Data Compression Conference, 201
Performance comparison of video encoders in light field image compression
Efficient compression plays a significant role in Light Fieldimaging technology because of the huge amount of data neededfor their representation. Video encoders using different strategiesare commonly used for Light Field image compression. In this pa-per, different video encoder implementations including HM, VTM,x265, xvc, VP9, and AV1 are analysed and compared in termsof coding efficiency, and encoder/decoder time-complexity. Lightfield images are compressed as pseudo-videos
RLFC: Random Access Light Field Compression using Key Views and Bounded Integer Encoding
We present a new hierarchical compression scheme for encoding light field
images (LFI) that is suitable for interactive rendering. Our method (RLFC)
exploits redundancies in the light field images by constructing a tree
structure. The top level (root) of the tree captures the common high-level
details across the LFI, and other levels (children) of the tree capture
specific low-level details of the LFI. Our decompressing algorithm corresponds
to tree traversal operations and gathers the values stored at different levels
of the tree. Furthermore, we use bounded integer sequence encoding which
provides random access and fast hardware decoding for compressing the blocks of
children of the tree. We have evaluated our method for 4D two-plane
parameterized light fields. The compression rates vary from 0.08 - 2.5 bits per
pixel (bpp), resulting in compression ratios of around 200:1 to 20:1 for a PSNR
quality of 40 to 50 dB. The decompression times for decoding the blocks of LFI
are 1 - 3 microseconds per channel on an NVIDIA GTX-960 and we can render new
views with a resolution of 512X512 at 200 fps. Our overall scheme is simple to
implement and involves only bit manipulations and integer arithmetic
operations.Comment: Accepted for publication at Symposium on Interactive 3D Graphics and
Games (I3D '19
Joint exploration model based light field image coding: A comparative study
© 2017 IEEE. The recent light field imaging technology has been attracting a lot of interests due to its potential applications in a large number of areas including Virtual Reality, Augmented Reality (VR/AR), Teleconferencing, and E-learning. Light Field (LF) data is able to provide rich visual information such as scene rendering with changes in depth of field, viewpoint, and focal length. However, Light Field data usually associates to a critical problem - the massive data. Therefore, compressing LF data is one of the main challenges in LF research. In this context, we present in this paper a comparative study for compressing LF data with not only the widely used image/video coding standards, such as JPEG-2000, H.264/AVC, HEVC and Google/VP9 but also with the most recent image/video coding solution, the Joint Exploration Model. In addition, this paper also proposes a LF image coding flow, which can be used as a benchmark for future LF compression evaluation. Finally, the compression efficiency of these coding solutions is thoroughly compared throughout a rich set of test conditions
Scalable light field coding with support for region of interest enhancement
Light field imaging based on microlens arrays - a.k.a. holoscopic, plenoptic, and integral imaging - has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representation and coding solutions that support emerging manipulation and interaction functionalities. In this context, this paper proposes a novel scalable coding approach that supports a new type of scalability, referred to as Field of View (FOV) scalability, in which enhancement layers can correspond to regions of interest (ROI). The proposed scalable coding approach comprises a base layer compliant with the High Efficiency Video Coding (HEVC) standard, complemented by one or more enhancement layers that progressively allow richer versions of the same light field content in terms of content manipulation and interaction possibilities, for the whole scene or just for a given ROI. Experimental results show the advantages of the proposed scalable coding approach with ROI support to cater for users with different preferences/requirements in terms of interaction functionalities.info:eu-repo/semantics/acceptedVersio
Optimized reference picture selection for light field image coding
This paper proposes a new reference picture selection method for light field image coding using the pseudo-video sequence (PVS) format. State-of-the-art solutions to encode light field images using the PVS format rely on video coding standards to exploit the inter-view redundancy between each sub-aperture image (SAI) that composes the light field. However, the PVS scanning order is not usually considered by the video codec. The proposed solution signals the PVS scanning order to the decoder, enabling implicit optimized reference picture selection for each specific scanning order. With the proposed method each reference picture is selected by minimizing the Euclidean distance to the current SAI being encoded. Experimental results show that, for the same PVS scanning order, the proposed optimized reference picture selection codec outperforms HEVC video coding standard for light field image coding, up to 50% in terms of bitrate savings.info:eu-repo/semantics/acceptedVersio
Light field image processing : overview and research issues
Light field (LF) imaging first appeared in the computer graphics community with the goal of photorealistic 3D rendering [1]. Motivated by a variety of potential applications in various domains (e.g., computational photography, augmented reality, light field microscopy, medical imaging, 3D robotic, particle image velocimetry), imaging from real light fields has recently gained in popularity, both at the research and industrial level.peer-reviewe
Light Field Compression by Residual CNN Assisted JPEG
Light field (LF) imaging has gained significant attention due to its recent
success in 3-dimensional (3D) displaying and rendering as well as augmented and
virtual reality usage. Nonetheless, because of the two extra dimensions, LFs
are much larger than conventional images. We develop a JPEG-assisted
learning-based technique to reconstruct an LF from a JPEG bitstream with a bit
per pixel ratio of 0.0047 on average. For compression, we keep the LF's center
view and use JPEG compression with 50% quality. Our reconstruction pipeline
consists of a small JPEG enhancement network (JPEG-Hance), a depth estimation
network (Depth-Net), followed by view synthesizing by warping the enhanced
center view. Our pipeline is significantly faster than using video compression
on pseudo-sequences extracted from an LF, both in compression and
decompression, while maintaining effective performance. We show that with a 1%
compression time cost and 18x speedup for decompression, our methods
reconstructed LFs have better structural similarity index metric (SSIM) and
comparable peak signal-to-noise ratio (PSNR) compared to the state-of-the-art
video compression techniques used to compress LFs
Light field image coding using high order prediction training
This paper proposes a new method for light field image coding relying on a high order prediction mode based on a training algorithm. The proposed approach is applied as an Intra prediction method based on a two-stage block-wise high order prediction model that supports geometric transformations up to eight degrees of freedom. Light field images comprise an array of micro-images that are related by complex perspective deformations that cannot be efficiently compensated by state-of-the-art image coding techniques, which are usually based on low order translational prediction models. The proposed prediction mode is able to exploit the non-local spatial redundancy introduced by light field image structure and a training algorithm is applied on different micro-images that are available in the reference region aiming at reducing the amount of signaling data sent to the receiver. The training direction that generates the most efficient geometric transformation for the current block is determined in the encoder side and signaled to the decoder using an index. The decoder is therefore able to repeat the high order prediction training to generate the desired geometric transformation. Experimental results show bitrate savings up to 12.57% and 50.03% relatively to a light field image coding solution based on low order prediction without training and HEVC, respectively.info:eu-repo/semantics/acceptedVersio