4 research outputs found
Flexible Neural Image Compression via Code Editing
Neural image compression (NIC) has outperformed traditional image codecs in
rate-distortion (R-D) performance. However, it usually requires a dedicated
encoder-decoder pair for each point on R-D curve, which greatly hinders its
practical deployment. While some recent works have enabled bitrate control via
conditional coding, they impose strong prior during training and provide
limited flexibility. In this paper we propose Code Editing, a highly flexible
coding method for NIC based on semi-amortized inference and adaptive
quantization. Our work is a new paradigm for variable bitrate NIC. Furthermore,
experimental results show that our method surpasses existing variable-rate
methods, and achieves ROI coding and multi-distortion trade-off with a single
decoder.Comment: NeurIPS 202
Conditional Perceptual Quality Preserving Image Compression
We propose conditional perceptual quality, an extension of the perceptual
quality defined in \citet{blau2018perception}, by conditioning it on user
defined information. Specifically, we extend the original perceptual quality
to the conditional perceptual quality
, where is the original image, is the
reconstructed, is side information defined by user and is
divergence. We show that conditional perceptual quality has similar theoretical
properties as rate-distortion-perception trade-off \citep{blau2019rethinking}.
Based on these theoretical results, we propose an optimal framework for
conditional perceptual quality preserving compression. Experimental results
show that our codec successfully maintains high perceptual quality and semantic
quality at all bitrate. Besides, by providing a lowerbound of common randomness
required, we settle the previous arguments on whether randomness should be
incorporated into generator for (conditional) perceptual quality compression.
The source code is provided in supplementary material
Bit Allocation using Optimization
In this paper, we consider the problem of bit allocation in neural video
compression (NVC). Due to the frame reference structure, current NVC methods
using the same R-D (Rate-Distortion) trade-off parameter for all
frames are suboptimal, which brings the need for bit allocation. Unlike
previous methods based on heuristic and empirical R-D models, we propose to
solve this problem by gradient-based optimization. Specifically, we first
propose a continuous bit implementation method based on Semi-Amortized
Variational Inference (SAVI). Then, we propose a pixel-level implicit bit
allocation method using iterative optimization by changing the SAVI target.
Moreover, we derive the precise R-D model based on the differentiable trait of
NVC. And we show the optimality of our method by proofing its equivalence to
the bit allocation with precise R-D model. Experimental results show that our
approach significantly improves NVC methods and outperforms existing bit
allocation methods. Our approach is plug-and-play for all differentiable NVC
methods, and it can be directly adopted on existing pre-trained models