7 research outputs found
Flexible Neural Image Compression via Code Editing
Neural image compression (NIC) has outperformed traditional image codecs in
rate-distortion (R-D) performance. However, it usually requires a dedicated
encoder-decoder pair for each point on R-D curve, which greatly hinders its
practical deployment. While some recent works have enabled bitrate control via
conditional coding, they impose strong prior during training and provide
limited flexibility. In this paper we propose Code Editing, a highly flexible
coding method for NIC based on semi-amortized inference and adaptive
quantization. Our work is a new paradigm for variable bitrate NIC. Furthermore,
experimental results show that our method surpasses existing variable-rate
methods, and achieves ROI coding and multi-distortion trade-off with a single
decoder.Comment: NeurIPS 202
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling
Reconstructing a 3D shape based on a single sketch image is challenging due
to the large domain gap between a sparse, irregular sketch and a regular, dense
3D shape. Existing works try to employ the global feature extracted from sketch
to directly predict the 3D coordinates, but they usually suffer from losing
fine details that are not faithful to the input sketch. Through analyzing the
3D-to-2D projection process, we notice that the density map that characterizes
the distribution of 2D point clouds (i.e., the probability of points projected
at each location of the projection plane) can be used as a proxy to facilitate
the reconstruction process. To this end, we first translate a sketch via an
image translation network to a more informative 2D representation that can be
used to generate a density map. Next, a 3D point cloud is reconstructed via a
two-stage probabilistic sampling process: first recovering the 2D points (i.e.,
the x and y coordinates) by sampling the density map; and then predicting the
depth (i.e., the z coordinate) by sampling the depth values at the ray
determined by each 2D point. Extensive experiments are conducted, and both
quantitative and qualitative results show that our proposed approach
significantly outperforms other baseline methods.Comment: 16 pages, 7 figures, accepted by ECCV 202
Bit Allocation using Optimization
In this paper, we consider the problem of bit allocation in neural video
compression (NVC). Due to the frame reference structure, current NVC methods
using the same R-D (Rate-Distortion) trade-off parameter for all
frames are suboptimal, which brings the need for bit allocation. Unlike
previous methods based on heuristic and empirical R-D models, we propose to
solve this problem by gradient-based optimization. Specifically, we first
propose a continuous bit implementation method based on Semi-Amortized
Variational Inference (SAVI). Then, we propose a pixel-level implicit bit
allocation method using iterative optimization by changing the SAVI target.
Moreover, we derive the precise R-D model based on the differentiable trait of
NVC. And we show the optimality of our method by proofing its equivalence to
the bit allocation with precise R-D model. Experimental results show that our
approach significantly improves NVC methods and outperforms existing bit
allocation methods. Our approach is plug-and-play for all differentiable NVC
methods, and it can be directly adopted on existing pre-trained models