Finite element based interpolation methods for spatial and temporal resolution enhancement for image sequences

Spatial resolution enhancement is a process for reconstructing a high resolution image from a low resolution image, whereas temporal resolution enhancement of encoded video aims of interpolating the skipped frames, making use of two successively received frames. In this thesis, a new image interpolation model, called the generalized image interpolation model , is developed in order to devise new techniques for spatial resolution enhancement of images, and temporal resolution enhancement of encoded video sequences. The interpolation model is based on the finite element method, and takes into account the unknown neighboring pixels, and therefore is capable of interpolating a collection of unknown pixels with an arbitrary shape, while providing a spatial continuity between the unknown pixels. Based on the generalized interpolation model, an edge-preserving iterative refinement scheme for spatial resolution enhancement of images is proposed. This scheme exploits not only the neighboring pixels whose values are known, but also takes into account those with unknown values. It is shown that the edge-preserving iterative refinement process maintains the smooth variation along a dominant edge in the up-scaled image. Simulation results show that the proposed scheme results in up-scaled images with subjective and objective qualities, which are better than those of the existing interpolation schemes. Further, the scheme is also shown to be capable of up-scaling an image by an arbitrary magnification factor, without resorting to extra steps, or the use of any conventional interpolation method. Next, error concealment-based MCI schemes are also presented for temporal resolution enhancement of encoded video sequences. These schemes are also based on the generalized image interpolation model, and need no pixel classification, thus reducing substantially the computational complexity. They are shown to be capable of concealing the errors in the homogeneous regions as well as in regions containing sharp edges. Experiments are carried out showing that the proposed schemes result in reconstructed frames having a better visual quality and a lower computational complexity than that provided by the existing techniques

A Bayesian Approach To Error Concealment In Encoded Video Streams

In ATM networks cell loss causes data to be dropped in the channel. When digital video is transmitted over these networks one must be able to reconstruct the missing data so that the impact of these errors is minimized. In this paper we describe a Bayesian approach to conceal these errors. Assuming that the digital video has been encoded using the MPEG1 or MPEG2 compression scheme, each frame is modeled as a Markov Random Field. A maximum a posteriori estimate of the missing macroblocks and motion vectors is described based on the model. 1. INTRODUCTION Broadband networks support a variety of applications involving high-resolution video and images. Asynchronous Transfer Mode (ATM) is expected to be the target communication protocol for these networks. To e#ectively transmit video tra#c over these networks, issues involved in packetizing encoded video sequences need to be studied. In particular it is important to study the e#ect of ATM cell loss, and develop post processing techniques t..