Video error concealment: a brief presentation

Typical error control techniques are not very well suited for video transmission. On the other hand, video transmission over error prone channels has increased greatly, e.g., over IP and wireless networks. These two facts combined together provided the necessary motivation for the development of a new set of techniques (error concealment) capable of dealing with transmission errors in video systems. These techniques can be categorized according with the approach they take to solve the problem. This categorization is presented, describing the assumptions in which they are based and giving a few examples in each category. Finally, the advantages and disadvantages of each category are presented

Detection of dirt impairments from archived film sequences : survey and evaluations

Film dirt is the most commonly encountered artifact in archive restoration applications. Since dirt usually appears as a temporally impulsive event, motion-compensated interframe processing is widely applied for its detection. However, motion-compensated prediction requires a high degree of complexity and can be unreliable when motion estimation fails. Consequently, many techniques using spatial or spatiotemporal filtering without motion were also been proposed as alternatives. A comprehensive survey and evaluation of existing methods is presented, in which both qualitative and quantitative performances are compared in terms of accuracy, robustness, and complexity. After analyzing these algorithms and identifying their limitations, we conclude with guidance in choosing from these algorithms and promising directions for future research

Enhanced low bitrate H.264 video coding using decoder-side super-resolution and frame interpolation

Advanced inter-prediction modes are introduced recently in literature to improve video coding performances of both H.264 and High Efficiency Video Coding standards. Decoder-side motion analysis and motion vector derivation are proposed to reduce coding costs of motion information. Here, we introduce enhanced skip and direct modes for H.264 coding using decoder-side super-resolution (SR) and frame interpolation. P-and B-frames are downsampled and H.264 encoded at lower resolution (LR). Then reconstructed LR frames are super-resolved using decoder-side motion estimation. Alternatively for B-frames, bidirectional true motion estimation is performed to synthesize a B-frame from its reference frames. For P-frames, bicubic interpolation of the LR frame is used as an alternative to SR reconstruction. A rate-distortion optimal mode selection algorithm is developed to decide for each MB which of the two reconstructions to use as skip/direct mode prediction. Simulations indicate an average of 1.04 dB peak signal-to-noise ratio (PSNR) improvement or 23.0% bitrate reduction at low bitrates when compared with H.264 standard. The PSNR gains reach as high as 3.00 dB for inter-predicted frames and 3.78 dB when only B-frames are considered. Decoded videos exhibit significantly better visual quality as well.This research was supported by TUBITAK Career Grant 108E201Publisher's Versio

A Scalable Multiple Description Scheme for 3D Video Coding Based on the Interlayer Prediction Structure

The most recent literature indicates multiple description coding (MDC) as a promising coding approach to handle the problem of video transmission over unreliable networks with different quality and bandwidth constraints. Furthermore, following recent commercial availability of autostereoscopic 3D displays that allow 3D visual data to be viewed without the use of special headgear or glasses, it is anticipated that the applications of 3D video will increase rapidly in the near future. Moving from the concept of spatial MDC, in this paper we introduce some efficient algorithms to obtain 3D substreams that also exploit some form of scalability. These algorithms are then applied to both coded stereo sequences and to depth image-based rendering (DIBR). In these algorithms, we first generate four 3D subsequences by subsampling, and then two of these subsequences are jointly used to form each of the two descriptions. For each description, one of the original subsequences is predicted from the other one via some scalable algorithms, focusing on the inter layer prediction scheme. The proposed algorithms can be implemented as pre- and postprocessing of the standard H.264/SVC coder that remains fully compatible with any standard coder. The experimental results presented show that these algorithms provide excellent results

Synthetic Aperture Radar (SAR) data processing

The available and optimal methods for generating SAR imagery for NASA applications were identified. The SAR image quality and data processing requirements associated with these applications were studied. Mathematical operations and algorithms required to process sensor data into SAR imagery were defined. The architecture of SAR image formation processors was discussed, and technology necessary to implement the SAR data processors used in both general purpose and dedicated imaging systems was addressed

No-reference analysis of decoded MPEG images for PSNR estimation and post-processing

We propose no-reference analysis and processing of DCT (Discrete Cosine Transform) coded images based on estimation of selected MPEG parameters from the decoded video. The goal is to assess MPEG video quality and perform post-processing without access to neither the original stream nor the code stream. Solutions are presented for MPEG-2 video. A method to estimate the quantization parameters of DCT coded images and MPEG I-frames at the macro-block level is presented. The results of this analysis is used for deblocking and deringing artifact reduction and no-reference PSNR estimation without code stream access. An adaptive deringing method using texture classification is presented. On the test set, the quantization parameters in MPEG-2 I-frames are estimated with an overall accuracy of 99.9% and the PSNR is estimated with an overall average error of 0.3dB. The deringing and deblocking algorithms yield improvements of 0.3dB on the MPEG-2 decoded test sequences

Adaptive delivery of real-time streaming video

Thesis (M.Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2001.Includes bibliographical references (p. 87-92).While there is an increasing demand for streaming video applications on the Internet, various network characteristics make the deployment of these applications more challenging than traditional Internet applications like email and the Web. The applications that transmit data over the Internet must cope with the time-varying bandwidth and delay characteristics of the Internet and must be resilient to packet loss. This thesis examines these challenges and presents a system design and implementation that ameliorates some of the important problems with video streaming over the Internet. Video sequences are typically compressed in a format such as MPEG-4 to achieve bandwidth efficiency. Video compression exploits redundancy between frames to achieve higher compression. However, packet loss can be detrimental to compressed video with interdependent frames because errors potentially propagate across many frames. While the need for low latency prevents the retransmission of all lost data, we leverage the characteristics of MPEG-4 to selectively retransmit only the most important data in order to limit the propagation of errors. We quantify the effects of packet loss on the quality of MPEG-4 video, develop an analytical model to explain these effects, and present an RTP-compatible protocol-which we call SR-RTP--to adaptively deliver higher quality video in the face of packet loss. The Internet's variable bandwidth and delay make it difficult to achieve high utilization, Tcp friendliness, and a high-quality constant playout rate; a video streaming system should adapt to these changing conditions and tailor the quality of the transmitted bitstream to available bandwidth. Traditional congestion avoidance schemes such as TCP's additive-increase/multiplicative/decrease (AIMD) cause large oscillations in transmission rates that degrade the perceptual quality of the video stream. To combat bandwidth variation, we design a scheme for performing quality adaptation of layered video for a general family of congestion control algorithms called binomial congestion control and show that a combination of smooth congestion control and clever receiver-buffered quality adaptation can reduce oscillations, increase interactivity, and deliver higher quality video for a given amount of buffering. We have integrated this selective reliability and quality adaptation into a publicly available software library. Using this system as a testbed, we show that the use of selective reliability can greatly increase the quality of received video, and that the use of binomial congestion control and receiver quality adaptation allow for increased user interactivity and better video quality.by Nicholas G. Feamster.M.Eng