523 research outputs found
Survey of Error Concealment techniques: Research directions and open issues
© 2015 IEEE. Error Concealment (EC) techniques use either spatial, temporal or a combination of both types of information to recover the data lost in transmitted video. In this paper, existing EC techniques are reviewed, which are divided into three categories, namely Intra-frame EC, Inter-frame EC, and Hybrid EC techniques. We first focus on the EC techniques developed for the H.264/AVC standard. The advantages and disadvantages of these EC techniques are summarized with respect to the features in H.264. Then, the EC algorithms are also analyzed. These EC algorithms have been recently adopted in the newly introduced H.265/HEVC standard. A performance comparison between the classic EC techniques developed for H.264 and H.265 is performed in terms of the average PSNR. Lastly, open issues in the EC domain are addressed for future research consideration
User-Oriented QoS in Packet Video Delivery
We focus on packet video delivery, with an emphasis on the quality of service perceived by the end-user. A video signal passes through several subsystems, such as the source coder, the network and the decoder. Each of these can impair the information, either by data loss or by introducing delay. We describe how each of the subsystems can be tuned to optimize the quality of the delivered signal, for a given available bit rate in the network. The assessment of end-user quality is not trivial. We present recent research results, which rely on a model of the human visual system
Machine Learning for Multimedia Communications
Machine learning is revolutionizing the way multimedia information is processed and transmitted to users. After intensive and powerful training, some impressive efficiency/accuracy improvements have been made all over the transmission pipeline. For example, the high model capacity of the learning-based architectures enables us to accurately model the image and video behavior such that tremendous compression gains can be achieved. Similarly, error concealment, streaming strategy or even user perception modeling have widely benefited from the recent learningoriented developments. However, learning-based algorithms often imply drastic changes to the way data are represented or consumed, meaning that the overall pipeline can be affected even though a subpart of it is optimized. In this paper, we review the recent major advances that have been proposed all across the transmission chain, and we discuss their potential impact and the research challenges that they raise
Loss-resilient Coding of Texture and Depth for Free-viewpoint Video Conferencing
Free-viewpoint video conferencing allows a participant to observe the remote
3D scene from any freely chosen viewpoint. An intermediate virtual viewpoint
image is commonly synthesized using two pairs of transmitted texture and depth
maps from two neighboring captured viewpoints via depth-image-based rendering
(DIBR). To maintain high quality of synthesized images, it is imperative to
contain the adverse effects of network packet losses that may arise during
texture and depth video transmission. Towards this end, we develop an
integrated approach that exploits the representation redundancy inherent in the
multiple streamed videos a voxel in the 3D scene visible to two captured views
is sampled and coded twice in the two views. In particular, at the receiver we
first develop an error concealment strategy that adaptively blends
corresponding pixels in the two captured views during DIBR, so that pixels from
the more reliable transmitted view are weighted more heavily. We then couple it
with a sender-side optimization of reference picture selection (RPS) during
real-time video coding, so that blocks containing samples of voxels that are
visible in both views are more error-resiliently coded in one view only, given
adaptive blending will erase errors in the other view. Further, synthesized
view distortion sensitivities to texture versus depth errors are analyzed, so
that relative importance of texture and depth code blocks can be computed for
system-wide RPS optimization. Experimental results show that the proposed
scheme can outperform the use of a traditional feedback channel by up to 0.82
dB on average at 8% packet loss rate, and by as much as 3 dB for particular
frames
Error resilient packet switched H.264 video telephony over third generation networks.
Real-time video communication over wireless networks is a challenging problem because
wireless channels suffer from fading, additive noise and interference, which translate
into packet loss and delay. Since modern video encoders deliver video packets with
decoding dependencies, packet loss and delay can significantly degrade the video quality
at the receiver. Many error resilience mechanisms have been proposed to combat packet
loss in wireless networks, but only a few were specifically designed for packet switched
video telephony over Third Generation (3G) networks.
The first part of the thesis presents an error resilience technique for packet switched
video telephony that combines application layer Forward Error Correction (FEC) with
rateless codes, Reference Picture Selection (RPS) and cross layer optimization. Rateless
codes have lower encoding and decoding computational complexity compared to traditional
error correcting codes. One can use them on complexity constrained hand-held
devices. Also, their redundancy does not need to be fixed in advance and any number of
encoded symbols can be generated on the fly. Reference picture selection is used to limit
the effect of spatio-temporal error propagation. Limiting the effect of spatio-temporal
error propagation results in better video quality. Cross layer optimization is used to
minimize the data loss at the application layer when data is lost at the data link layer.
Experimental results on a High Speed Packet Access (HSPA) network simulator for
H.264 compressed standard video sequences show that the proposed technique achieves
significant Peak Signal to Noise Ratio (PSNR) and Percentage Degraded Video Duration
(PDVD) improvements over a state of the art error resilience technique known as
Interactive Error Control (IEC), which is a combination of Error Tracking and feedback
based Reference Picture Selection. The improvement is obtained at a cost of higher
end-to-end delay.
The proposed technique is improved by making the FEC (Rateless code) redundancy
channel adaptive. Automatic Repeat Request (ARQ) is used to adjust the redundancy
of the Rateless codes according to the channel conditions. Experimental results show
that the channel adaptive scheme achieves significant PSNR and PDVD improvements
over the static scheme for a simulated Long Term Evolution (LTE) network.
In the third part of the thesis, the performance of the previous two schemes is
improved by making the transmitter predict when rateless decoding will fail. In this
case, reference picture selection is invoked early and transmission of encoded symbols
for that source block is aborted. Simulations for an LTE network show that this results
in video quality improvement and bandwidth savings.
In the last part of the thesis, the performance of the adaptive technique is improved
by exploiting the history of the wireless channel. In a Rayleigh fading wireless channel,
the RLC-PDU losses are correlated under certain conditions. This correlation is
exploited to adjust the redundancy of the Rateless code and results in higher Rateless
code decoding success rate and higher video quality. Simulations for an LTE network
show that the improvement was significant when the packet loss rate in the two wireless
links was 10%.
To facilitate the implementation of the proposed error resilience techniques in practical
scenarios, RTP/UDP/IP level packetization schemes are also proposed for each
error resilience technique.
Compared to existing work, the proposed error resilience techniques provide better
video quality. Also, more emphasis is given to implementation issues in 3G networks
- …