Multiview 3D Video Denoising in Sliding 3D DCT Domain

With the widespread interest in 3D technology areas such as displays, cameras, and processing, the 3D video is becoming widely available. Due to correlation between views in multiview 3D video at the same temporal location, it is possible to perform video processing operations more efficiently comparing to regular 2D video. In order to improve denoising performance for multiview video, we propose an algorithm based on denoising in 3D DCT domain, which is competitive in performance with state-of-art denoising algorithms and it is suitable for real-time implementation. The proposed algorithm searches for corresponding image patches in temporal and inter-view directions, selects 8 patches with lowest dissimilarity measure, and performs denoising in 3D DCT domain. The novel interview image patch search method brings up to 1.62dB gain in terms of average luma Peak Signal-to-Noise Ratio (PSNR), with average gain 0.6 - 0.8 dB depending on the amount of noise present in test sequences.Peer reviewe

An online algorithm for separating sparse and low-dimensional signal sequences from their sum, and its applications in video processing

In signal processing, ``low-rank + sparse\u27\u27 is an important assumption when separating two signals from their sum. Many applications, e.g., video foreground/background separation are well-formulated by this assumption. In this work, with the ``low-rank + sparse\u27\u27 assumption, we design and evaluate an online algorithm, called practical recursive projected compressive sensing (prac-ReProCS) for recovering a time sequence of sparse vectors St and a time sequence of dense vectors Lt from their sum, Mt = St + Lt, when the Lt\u27s lie in a slowly changing low-dimensional subspace of the full space. In the first part of this work (Chapter 1-5), we study and discuss the prac-ReProCS algorithm, the practical version of the original ReProCS algorithm. We apply prac-ReProCS to a key application -- video layering, where the goal is to separate a video sequence into a slowly changing background sequence and a sparse foreground sequence that consists of one or more moving regions/objects on-the-fly. Via experiments we show that prac-ReProCS has significantly better performance compared with other state-of-the-art robust-pca methods when applied to video foreground-background separation. In the second part of this work (Chapter 6), we study the problem of video denoising. We apply prac-ReProCS to video denoising as a preprocessing step. We develop a novel approach to video denoising that is based on the idea that many noisy or corrupted videos can be split into three parts -- the ``low-rank laye\u27\u27, the ``sparse layer\u27\u27 and a small residual which is small and bounded. We show using extensive experiments, layering-then-denoising is effective, especially for long videos with small-sized images that those corrupted by general large variance noise or by large sparse noise, e.g., salt-and-pepper noise. In the last part of this work (Chapter 7), we discuss an independent problem called logo detection and propose a future research direction where prac-ReProCS can be combined with deep learning solutions

Efficient architectures for multidimensional discrete transforms in image and video processing applications

PhD ThesisThis thesis introduces new image compression algorithms, their related architectures and data transforms architectures. The proposed architectures consider the current hardware architectures concerns, such as power consumption, hardware usage, memory requirement, computation time and output accuracy. These concerns and problems are crucial in multidimensional image and video processing applications. This research is divided into three image and video processing related topics: low complexity non-transform-based image compression algorithms and their architectures, architectures for multidimensional Discrete Cosine Transform (DCT); and architectures for multidimensional Discrete Wavelet Transform (DWT). The proposed architectures are parameterised in terms of wordlength, pipelining and input data size. Taking such parameterisation into account, efficient non-transform based and low complexity image compression algorithms for better rate distortion performance are proposed. The proposed algorithms are based on the Adaptive Quantisation Coding (AQC) algorithm, and they achieve a controllable output bit rate and accuracy by considering the intensity variation of each image block. Their high speed, low hardware usage and low power consumption architectures are also introduced and implemented on Xilinx devices. Furthermore, efficient hardware architectures for multidimensional DCT based on the 1-D DCT Radix-2 and 3-D DCT Vector Radix (3-D DCT VR) fast algorithms have been proposed. These architectures attain fast and accurate 3-D DCT computation and provide high processing speed and power consumption reduction. In addition, this research also introduces two low hardware usage 3-D DCT VR architectures. Such architectures perform the computation of butterfly and post addition stages without using block memory for data transposition, which in turn reduces the hardware usage and improves the performance of the proposed architectures. Moreover, parallel and multiplierless lifting-based architectures for the 1-D, 2-D and 3-D Cohen-Daubechies-Feauveau 9/7 (CDF 9/7) DWT computation are also introduced. The presented architectures represent an efficient multiplierless and low memory requirement CDF 9/7 DWT computation scheme using the separable approach. Furthermore, the proposed architectures have been implemented and tested using Xilinx FPGA devices. The evaluation results have revealed that a speed of up to 315 MHz can be achieved in the proposed AQC-based architectures. Further, a speed of up to 330 MHz and low utilisation rate of 722 to 1235 can be achieved in the proposed 3-D DCT VR architectures. In addition, in the proposed 3-D DWT architecture, the computation time of 3-D DWT for data size of 144×176×8-pixel is less than 0.33 ms. Also, a power consumption of 102 mW at 50 MHz clock frequency using 256×256-pixel frame size is achieved. The accuracy tests for all architectures have revealed that a PSNR of infinite can be attained