818 research outputs found
Multi-Frame Quality Enhancement for Compressed Video
The past few years have witnessed great success in applying deep learning to
enhance the quality of compressed image/video. The existing approaches mainly
focus on enhancing the quality of a single frame, ignoring the similarity
between consecutive frames. In this paper, we investigate that heavy quality
fluctuation exists across compressed video frames, and thus low quality frames
can be enhanced using the neighboring high quality frames, seen as Multi-Frame
Quality Enhancement (MFQE). Accordingly, this paper proposes an MFQE approach
for compressed video, as a first attempt in this direction. In our approach, we
firstly develop a Support Vector Machine (SVM) based detector to locate Peak
Quality Frames (PQFs) in compressed video. Then, a novel Multi-Frame
Convolutional Neural Network (MF-CNN) is designed to enhance the quality of
compressed video, in which the non-PQF and its nearest two PQFs are as the
input. The MF-CNN compensates motion between the non-PQF and PQFs through the
Motion Compensation subnet (MC-subnet). Subsequently, the Quality Enhancement
subnet (QE-subnet) reduces compression artifacts of the non-PQF with the help
of its nearest PQFs. Finally, the experiments validate the effectiveness and
generality of our MFQE approach in advancing the state-of-the-art quality
enhancement of compressed video. The code of our MFQE approach is available at
https://github.com/ryangBUAA/MFQE.gitComment: to appear in CVPR 201
Multiple description video coding for stereoscopic 3D
In this paper, we propose an MDC schemes for stereoscopic 3D video. In the literature, MDC has previously been applied in 2D video but not so much in 3D video. The proposed algorithm enhances the error resilience of the 3D video using the combination of even and odd frame based MDC while retaining good temporal prediction efficiency for video over error-prone networks. Improvements are made to the original even and odd frame MDC scheme by adding a controllable amount of side information to improve frame interpolation at the decoder. The side information is also sent according to the video sequence motion for further improvement. The performance of the proposed algorithms is evaluated in error free and error prone environments especially for wireless channels. Simulation results show improved performance using the proposed MDC at high error rates compared to the single description coding (SDC) and the original even and odd frame MDC
REGION-BASED ADAPTIVE DISTRIBUTED VIDEO CODING CODEC
The recently developed Distributed Video Coding (DVC) is typically suitable for the
applications where the conventional video coding is not feasible because of its
inherent high-complexity encoding. Examples include video surveillance usmg
wireless/wired video sensor network and applications using mobile cameras etc. With
DVC, the complexity is shifted from the encoder to the decoder.
The practical application of DVC is referred to as Wyner-Ziv video coding (WZ)
where an estimate of the original frame called "side information" is generated using
motion compensation at the decoder. The compression is achieved by sending only
that extra information that is needed to correct this estimation. An error-correcting
code is used with the assumption that the estimate is a noisy version of the original
frame and the rate needed is certain amount of the parity bits. The side information is
assumed to have become available at the decoder through a virtual channel. Due to
the limitation of compensation method, the predicted frame, or the side information, is
expected to have varying degrees of success. These limitations stem from locationspecific
non-stationary estimation noise. In order to avoid these, the conventional
video coders, like MPEG, make use of frame partitioning to allocate optimum coder
for each partition and hence achieve better rate-distortion performance. The same,
however, has not been used in DVC as it increases the encoder complexity.
This work proposes partitioning the considered frame into many coding units
(region) where each unit is encoded differently. This partitioning is, however, done at
the decoder while generating the side-information and the region map is sent over to
encoder at very little rate penalty. The partitioning allows allocation of appropriate
DVC coding parameters (virtual channel, rate, and quantizer) to each region. The
resulting regions map is compressed by employing quadtree algorithm and
communicated to the encoder via the feedback channel. The rate control in DVC is
performed by channel coding techniques (turbo codes, LDPC, etc.). The performance
of the channel code depends heavily on the accuracy of virtual channel model that models estimation error for each region. In this work, a turbo code has been used and
an adaptive WZ DVC is designed both in transform domain and in pixel domain. The
transform domain WZ video coding (TDWZ) has distinct superior performance as
compared to the normal Pixel Domain Wyner-Ziv (PDWZ), since it exploits the
'
spatial redundancy during the encoding. The performance evaluations show that the
proposed system is superior to the existing distributed video coding solutions.
Although the, proposed system requires extra bits representing the "regions map" to be
transmitted, fuut still the rate gain is noticeable and it outperforms the state-of-the-art
frame based DVC by 0.6-1.9 dB.
The feedback channel (FC) has the role to adapt the bit rate to the changing
'
statistics between the side infonmation and the frame to be encoded. In the
unidirectional scenario, the encoder must perform the rate control. To correctly
estimate the rate, the encoder must calculate typical side information. However, the
rate cannot be exactly calculated at the encoder, instead it can only be estimated. This
work also prbposes a feedback-free region-based adaptive DVC solution in pixel
domain based on machine learning approach to estimate the side information.
Although the performance evaluations show rate-penalty but it is acceptable
considering the simplicity of the proposed algorithm.
vii
Transport of video over partial order connections
A Partial Order and partial reliable Connection (POC) is an end-to-end transport connection authorized to deliver objects in an order that can differ from the transmitted one. Such a connection is also authorized to lose some objects. The POC concept is motivated by the fact that heterogeneous best-effort networks such as Internet are plagued by unordered delivery of packets and losses, which tax the performances of current applications and protocols. It has been shown, in several research works, that out of order delivery is able to alleviate (with respect to CO service) the use of end systemsâ communication resources. In this paper, the efficiency of out-of-sequence delivery on MPEG video streams processing is studied. Firstly, the transport constraints (in terms of order and reliability) that can be relaxed by MPEG video decoders, for improving video transport, are detailed. Then, we analyze the performance gain induced by this approach in terms of blocking times and recovered errors. We demonstrate that POC connections fill not only the conceptual gap between TCP and UDP but also provide real performance improvements for the transport of multimedia streams such MPEG video
- âŠ