Many common types of data can be represented as functions that map
coordinates to signal values, such as pixel locations to RGB values in the case
of an image. Based on this view, data can be compressed by overfitting a
compact neural network to its functional representation and then encoding the
network weights. However, most current solutions for this are inefficient, as
quantization to low-bit precision substantially degrades the reconstruction
quality. To address this issue, we propose overfitting variational Bayesian
neural networks to the data and compressing an approximate posterior weight
sample using relative entropy coding instead of quantizing and entropy coding
it. This strategy enables direct optimization of the rate-distortion
performance by minimizing the β-ELBO, and target different
rate-distortion trade-offs for a given network architecture by adjusting
β. Moreover, we introduce an iterative algorithm for learning prior
weight distributions and employ a progressive refinement process for the
variational posterior that significantly enhances performance. Experiments show
that our method achieves strong performance on image and audio compression
while retaining simplicity.Comment: Preprin