Postprocessing of images coded using block DCT at low bit rates.

Sun, Deqing.Thesis (M.Phil.)--Chinese University of Hong Kong, 2007.Includes bibliographical references (leaves 86-91).Abstracts in English and Chinese.Abstract --- p.i摘要 --- p.iiiContributions --- p.ivAcknowledgement --- p.viAbbreviations --- p.xviiiNotations --- p.xxiChapter 1 --- Introduction --- p.1Chapter 1.1 --- Image compression and postprocessing --- p.1Chapter 1.2 --- A brief review of postprocessing --- p.3Chapter 1.3 --- Objective and methodology of the research --- p.7Chapter 1.4 --- Thesis organization --- p.8Chapter 1.5 --- A note on publication --- p.9Chapter 2 --- Background Study --- p.11Chapter 2.1 --- Image models --- p.11Chapter 2.1.1 --- Minimum edge difference (MED) criterion for block boundaries --- p.12Chapter 2.1.2 --- van Beek's edge model for an edge --- p.15Chapter 2.1.3 --- Fields of experts (FoE) for an image --- p.16Chapter 2.2 --- Degradation models --- p.20Chapter 2.2.1 --- Quantization constraint set (QCS) and uniform noise --- p.21Chapter 2.2.2 --- Narrow quantization constraint set (NQCS) --- p.22Chapter 2.2.3 --- Gaussian noise --- p.22Chapter 2.2.4 --- Edge width enlargement after quantization --- p.25Chapter 2.3 --- Use of these models for postprocessing --- p.27Chapter 2.3.1 --- MED and edge models --- p.27Chapter 2.3.2 --- The FoE prior model --- p.27Chapter 3 --- Postprocessing using MED and edge models --- p.28Chapter 3.1 --- Blocking artifacts suppression by coefficient restoration --- p.29Chapter 3.1.1 --- AC coefficient restoration by MED --- p.29Chapter 3.1.2 --- General derivation --- p.31Chapter 3.2 --- Detailed algorithm --- p.34Chapter 3.2.1 --- Edge identification --- p.36Chapter 3.2.2 --- Region classification --- p.36Chapter 3.2.3 --- Edge reconstruction --- p.37Chapter 3.2.4 --- Image reconstruction --- p.37Chapter 3.3 --- Experimental results --- p.38Chapter 3.3.1 --- Results of the proposed method --- p.38Chapter 3.3.2 --- Comparison with one wavelet-based method --- p.39Chapter 3.4 --- On the global minimum of the edge difference . . --- p.41Chapter 3.4.1 --- The constrained minimization problem . . --- p.41Chapter 3.4.2 --- Experimental examination --- p.42Chapter 3.4.3 --- Discussions --- p.43Chapter 3.5 --- Conclusions --- p.44Chapter 4 --- Postprocessing by the MAP criterion using FoE --- p.49Chapter 4.1 --- The proposed method --- p.49Chapter 4.1.1 --- The MAP criterion --- p.49Chapter 4.1.2 --- The optimization problem --- p.51Chapter 4.2 --- Experimental results --- p.52Chapter 4.2.1 --- Setting algorithm parameters --- p.53Chapter 4.2.2 --- Results --- p.56Chapter 4.3 --- Investigation on the quantization noise model . . --- p.58Chapter 4.4 --- Conclusions --- p.61Chapter 5 --- Conclusion --- p.71Chapter 5.1 --- Contributions --- p.71Chapter 5.1.1 --- Extension of the DCCR algorithm --- p.71Chapter 5.1.2 --- Examination of the MED criterion --- p.72Chapter 5.1.3 --- Use of the FoE prior in postprocessing . . --- p.72Chapter 5.1.4 --- Investigation on the quantization noise model --- p.73Chapter 5.2 --- Future work --- p.73Chapter 5.2.1 --- Degradation model --- p.73Chapter 5.2.2 --- Efficient implementation of the MAP method --- p.74Chapter 5.2.3 --- Postprocessing of compressed video --- p.75Chapter A --- Detailed derivation of coefficient restoration --- p.76Chapter B --- Implementation details of the FoE prior --- p.81Chapter B.1 --- The FoE prior model --- p.81Chapter B.2 --- Energy function and its gradient --- p.83Chapter B.3 --- Conjugate gradient descent method --- p.84Bibliography --- p.8

Super resolution and dynamic range enhancement of image sequences

Camera producers try to increase the spatial resolution of a camera by reducing size of sites on sensor array. However, shot noise causes the signal to noise ratio drop as sensor sites get smaller. This fact motivates resolution enhancement to be performed through software. Super resolution (SR) image reconstruction aims to combine degraded images of a scene in order to form an image which has higher resolution than all observations. There is a demand for high resolution images in biomedical imaging, surveillance, aerial/satellite imaging and high-definition TV (HDTV) technology. Although extensive research has been conducted in SR, attention has not been given to increase the resolution of images under illumination changes. In this study, a unique framework is proposed to increase the spatial resolution and dynamic range of a video sequence using Bayesian and Projection onto Convex Sets (POCS) methods. Incorporating camera response function estimation into image reconstruction allows dynamic range enhancement along with spatial resolution improvement. Photometrically varying input images complicate process of projecting observations onto common grid by violating brightness constancy. A contrast invariant feature transform is proposed in this thesis to register input images with high illumination variation. Proposed algorithm increases the repeatability rate of detected features among frames of a video. Repeatability rate is increased by computing the autocorrelation matrix using the gradients of contrast stretched input images. Presented contrast invariant feature detection improves repeatability rate of Harris corner detector around %25 on average. Joint multi-frame demosaicking and resolution enhancement is also investigated in this thesis. Color constancy constraint set is devised and incorporated into POCS framework for increasing resolution of color-filter array sampled images. Proposed method provides fewer demosaicking artifacts compared to existing POCS method and a higher visual quality in final image

Sustav za adaptivno postprocesiranje videosignala

Inclusion of the latest research work to a real life applications and achieving adaptive video signal postprocessing is a complex and timely task. The proposed framework breaks the filtering process into several distinct phases and enables a researcher to move at a faster pace by narrowing the scope of the research and enabling seamless integration of this work into inherently adaptive filtering process. Some of the benefits of this modular and adaptive framework have been presented on the example of the MPEG-4 postprocessing improvements.Uključivanje najnovijih istraživačkih dostignuća u svakodnevne primjene i postizanje adaptivnog postprocesiranja videosignala zahtjevan je zadatak za koji je potrebno mnogo vremena. Predloženi sustav dijeli postupak filtriranja u nekoliko jasno odvojenih faza te na taj način sužava opseg svakog pojedinog istraživanja. Time je omogućena jednostavna integracija novih dostignuća u inherentno adaptivan proces filtriranja, što omogućava osjetno brži napredak u istraživanju. Neke prednosti ovog modularnog i adaptivnog sustava prikazane su na primjeru poboljšanja procesa postprocesiranja videosignala kodiranog prema MPEG-4 normi

Learning-based Wavelet-like Transforms For Fully Scalable and Accessible Image Compression

The goal of this thesis is to improve the existing wavelet transform with the aid of machine learning techniques, so as to enhance coding efficiency of wavelet-based image compression frameworks, such as JPEG 2000. In this thesis, we first propose to augment the conventional base wavelet transform with two additional learned lifting steps -- a high-to-low step followed by a low-to-high step. The high-to-low step suppresses aliasing in the low-pass band by using the detail bands at the same resolution, while the low-to-high step aims to further remove redundancy from detail bands by using the corresponding low-pass band. These two additional steps reduce redundancy (notably aliasing information) amongst the wavelet subbands, and also improve the visual quality of reconstructed images at reduced resolutions. To train these two networks in an end-to-end fashion, we develop a backward annealing approach to overcome the non-differentiability of the quantization and cost functions during back-propagation. Importantly, the two additional networks share a common architecture, named a proposal-opacity topology, which is inspired and guided by a specific theoretical argument related to geometric flow. This particular network topology is compact and with limited non-linearities, allowing a fully scalable system; one pair of trained network parameters are applied for all levels of decomposition and for all bit-rates of interest. By employing the additional lifting networks within the JPEG2000 image coding standard, we can achieve up to 17.4% average BD bit-rate saving over a wide range of bit-rates, while retaining the quality and resolution scalability features of JPEG2000. Built upon the success of the high-to-low and low-to-high steps, we then study more broadly the extension of neural networks to all lifting steps that correspond to the base wavelet transform. The purpose of this comprehensive study is to understand what is the most effective way to develop learned wavelet-like transforms for highly scalable and accessible image compression. Specifically, we examine the impact of the number of learned lifting steps, the number of layers and the number of channels in each learned lifting network, and kernel support in each layer. To facilitate the study, we develop a generic training methodology that is simultaneously appropriate to all lifting structures considered. Experimental results ultimately suggest that to improve the existing wavelet transform, it is more profitable to augment a larger wavelet transform with more diverse high-to-low and low-to-high steps, rather than developing deep fully learned lifting structures

Motion Segmentation Aided Super Resolution Image Reconstruction

This dissertation addresses Super Resolution (SR) Image Reconstruction focusing on motion segmentation. The main thrust is Information Complexity guided Gaussian Mixture Models (GMMs) for Statistical Background Modeling. In the process of developing our framework we also focus on two other topics; motion trajectories estimation toward global and local scene change detections and image reconstruction to have high resolution (HR) representations of the moving regions. Such a framework is used for dynamic scene understanding and recognition of individuals and threats with the help of the image sequences recorded with either stationary or non-stationary camera systems. We introduce a new technique called Information Complexity guided Statistical Background Modeling. Thus, we successfully employ GMMs, which are optimal with respect to information complexity criteria. Moving objects are segmented out through background subtraction which utilizes the computed background model. This technique produces superior results to competing background modeling strategies. The state-of-the-art SR Image Reconstruction studies combine the information from a set of unremarkably different low resolution (LR) images of static scene to construct an HR representation. The crucial challenge not handled in these studies is accumulating the corresponding information from highly displaced moving objects. In this aspect, a framework of SR Image Reconstruction of the moving objects with such high level of displacements is developed. Our assumption is that LR images are different from each other due to local motion of the objects and the global motion of the scene imposed by non-stationary imaging system. Contrary to traditional SR approaches, we employed several steps. These steps are; the suppression of the global motion, motion segmentation accompanied by background subtraction to extract moving objects, suppression of the local motion of the segmented out regions, and super-resolving accumulated information coming from moving objects rather than the whole scene. This results in a reliable offline SR Image Reconstruction tool which handles several types of dynamic scene changes, compensates the impacts of camera systems, and provides data redundancy through removing the background. The framework proved to be superior to the state-of-the-art algorithms which put no significant effort toward dynamic scene representation of non-stationary camera systems