6,113 research outputs found
Generative Joint Source-Channel Coding for Semantic Image Transmission
Recent works have shown that joint source-channel coding (JSCC) schemes using
deep neural networks (DNNs), called DeepJSCC, provide promising results in
wireless image transmission. However, these methods mostly focus on the
distortion of the reconstructed signals with respect to the input image, rather
than their perception by humans. However, focusing on traditional distortion
metrics alone does not necessarily result in high perceptual quality,
especially in extreme physical conditions, such as very low bandwidth
compression ratio (BCR) and low signal-to-noise ratio (SNR) regimes. In this
work, we propose two novel JSCC schemes that leverage the perceptual quality of
deep generative models (DGMs) for wireless image transmission, namely
InverseJSCC and GenerativeJSCC. While the former is an inverse problem approach
to DeepJSCC, the latter is an end-to-end optimized JSCC scheme. In both, we
optimize a weighted sum of mean squared error (MSE) and learned perceptual
image patch similarity (LPIPS) losses, which capture more semantic similarities
than other distortion metrics. InverseJSCC performs denoising on the distorted
reconstructions of a DeepJSCC model by solving an inverse optimization problem
using style-based generative adversarial network (StyleGAN). Our simulation
results show that InverseJSCC significantly improves the state-of-the-art
(SotA) DeepJSCC in terms of perceptual quality in edge cases. In
GenerativeJSCC, we carry out end-to-end training of an encoder and a
StyleGAN-based decoder, and show that GenerativeJSCC significantly outperforms
DeepJSCC both in terms of distortion and perceptual quality.Comment: 12 pages, 9 figure
- …