Finite element based interpolation methods for spatial and temporal resolution enhancement for image sequences

Spatial resolution enhancement is a process for reconstructing a high resolution image from a low resolution image, whereas temporal resolution enhancement of encoded video aims of interpolating the skipped frames, making use of two successively received frames. In this thesis, a new image interpolation model, called the generalized image interpolation model , is developed in order to devise new techniques for spatial resolution enhancement of images, and temporal resolution enhancement of encoded video sequences. The interpolation model is based on the finite element method, and takes into account the unknown neighboring pixels, and therefore is capable of interpolating a collection of unknown pixels with an arbitrary shape, while providing a spatial continuity between the unknown pixels. Based on the generalized interpolation model, an edge-preserving iterative refinement scheme for spatial resolution enhancement of images is proposed. This scheme exploits not only the neighboring pixels whose values are known, but also takes into account those with unknown values. It is shown that the edge-preserving iterative refinement process maintains the smooth variation along a dominant edge in the up-scaled image. Simulation results show that the proposed scheme results in up-scaled images with subjective and objective qualities, which are better than those of the existing interpolation schemes. Further, the scheme is also shown to be capable of up-scaling an image by an arbitrary magnification factor, without resorting to extra steps, or the use of any conventional interpolation method. Next, error concealment-based MCI schemes are also presented for temporal resolution enhancement of encoded video sequences. These schemes are also based on the generalized image interpolation model, and need no pixel classification, thus reducing substantially the computational complexity. They are shown to be capable of concealing the errors in the homogeneous regions as well as in regions containing sharp edges. Experiments are carried out showing that the proposed schemes result in reconstructed frames having a better visual quality and a lower computational complexity than that provided by the existing techniques

Temporal resolution enhancement in compressed video sequences

ABSTRACT — Compressed video may possess a number of artifacts, both spatial and temporal. Spatial compression artifacts arise as a result of quantization of the transform-domain coefficients, and are often manifested as blocking and ringing artifacts. Temporal limitations in compressed video occur when the encoder, in an effort to reduce bandwidth, drops frames. Omitting frames decreases the reconstructed frame rate, which can cause motion to appear jerky and uneven. This paper discusses a method to increase the frame rate of video compressed with the DCT by inserting images between received frames of the sequence. The Bayesian formulation of the restoration prevents spatial compression artifacts in the received frames from propagating to the reconstructed frames