38 research outputs found
Communication Beyond Transmitting Bits: Semantics-Guided Source and Channel Coding
Classical communication paradigms focus on accurately transmitting bits over
a noisy channel, and Shannon theory provides a fundamental theoretical limit on
the rate of reliable communications. In this approach, bits are treated
equally, and the communication system is oblivious to what meaning these bits
convey or how they would be used. Future communications towards intelligence
and conciseness will predictably play a dominant role, and the proliferation of
connected intelligent agents requires a radical rethinking of coded
transmission paradigm to support the new communication morphology on the
horizon. The recent concept of "semantic communications" offers a promising
research direction. Injecting semantic guidance into the coded transmission
design to achieve semantics-aware communications shows great potential for
further breakthrough in effectiveness and reliability. This article sheds light
on semantics-guided source and channel coding as a transmission paradigm of
semantic communications, which exploits both data semantics diversity and
wireless channel diversity together to boost the whole system performance. We
present the general system architecture and key techniques, and indicate some
open issues on this topic.Comment: IEEE Wireless Communications, text overlap with arXiv:2112.0309
Improved Nonlinear Transform Source-Channel Coding to Catalyze Semantic Communications
Recent deep learning methods have led to increased interest in solving
high-efficiency end-to-end transmission problems. These methods, we call
nonlinear transform source-channel coding (NTSCC), extract the semantic latent
features of source signal, and learn entropy model to guide the joint
source-channel coding with variable rate to transmit latent features over
wireless channels. In this paper, we propose a comprehensive framework for
improving NTSCC, thereby higher system coding gain, better model versatility,
and more flexible adaptation strategy aligned with semantic guidance are all
achieved. This new sophisticated NTSCC model is now ready to support large-size
data interaction in emerging XR, which catalyzes the application of semantic
communications. Specifically, we propose three useful improvement approaches.
First, we introduce a contextual entropy model to better capture the spatial
correlations among the semantic latent features, thereby more accurate rate
allocation and contextual joint source-channel coding are developed accordingly
to enable higher coding gain. On that basis, we further propose response
network architectures to formulate versatile NTSCC, i.e., once-trained model
supports various rates and channel states that benefits the practical
deployment. Following this, we propose an online latent feature editing method
to enable more flexible coding rate control aligned with some specific semantic
guidance. By comprehensively applying the above three improvement methods for
NTSCC, a deployment-friendly semantic coded transmission system stands out
finally. Our improved NTSCC system has been experimentally verified to achieve
considerable bandwidth saving versus the state-of-the-art engineered VTM + 5G
LDPC coded transmission system with lower processing latency
Adaptive Semantic Communications: Overfitting the Source and Channel for Profit
Most semantic communication systems leverage deep learning models to provide
end-to-end transmission performance surpassing the established source and
channel coding approaches. While, so far, research has mainly focused on
architecture and model improvements, but such a model trained over a full
dataset and ergodic channel responses is unlikely to be optimal for every test
instance. Due to limitations on the model capacity and imperfect optimization
and generalization, such learned models will be suboptimal especially when the
testing data distribution or channel response is different from that in the
training phase, as is likely to be the case in practice. To tackle this, in
this paper, we propose a novel semantic communication paradigm by leveraging
the deep learning model's overfitting property. Our model can for instance be
updated after deployment, which can further lead to substantial gains in terms
of the transmission rate-distortion (RD) performance. This new system is named
adaptive semantic communication (ASC). In our ASC system, the ingredients of
wireless transmitted stream include both the semantic representations of source
data and the adapted decoder model parameters. Specifically, we take the
overfitting concept to the extreme, proposing a series of ingenious methods to
adapt the semantic codec or representations to an individual data or channel
state instance. The whole ASC system design is formulated as an optimization
problem whose goal is to minimize the loss function that is a tripartite
tradeoff among the data rate, model rate, and distortion terms. The experiments
(including user study) verify the effectiveness and efficiency of our ASC
system. Notably, the substantial gain of our overfitted coding paradigm can
catalyze semantic communication upgrading to a new era
Wireless Deep Video Semantic Transmission
In this paper, we design a new class of high-efficiency deep joint
source-channel coding methods to achieve end-to-end video transmission over
wireless channels. The proposed methods exploit nonlinear transform and
conditional coding architecture to adaptively extract semantic features across
video frames, and transmit semantic feature domain representations over
wireless channels via deep joint source-channel coding. Our framework is
collected under the name deep video semantic transmission (DVST). In
particular, benefiting from the strong temporal prior provided by the feature
domain context, the learned nonlinear transform function becomes temporally
adaptive, resulting in a richer and more accurate entropy model guiding the
transmission of current frame. Accordingly, a novel rate adaptive transmission
mechanism is developed to customize deep joint source-channel coding for video
sources. It learns to allocate the limited channel bandwidth within and among
video frames to maximize the overall transmission performance. The whole DVST
design is formulated as an optimization problem whose goal is to minimize the
end-to-end transmission rate-distortion performance under perceptual quality
metrics or machine vision task performance metrics. Across standard video
source test sequences and various communication scenarios, experiments show
that our DVST can generally surpass traditional wireless video coded
transmission schemes. The proposed DVST framework can well support future
semantic communications due to its video content-aware and machine vision task
integration abilities.Comment: published in IEEE JSA