IMAGE AND VIDEO ENHANCEMENT USING SPARSE CODING, BELIEF PROPAGATION AND MATRIX COMPLETION

Super resolution as an exciting application in image processing was studied widely in the literature. This dissertation presents new approaches to video super resolution, based on sparse coding and belief propagation. First, find candidate match pixels on multiple frames using sparse coding and belief propagation. Second, incorporate information from these candidate pixels with weights computed using the Nonlocal-Means (NLM) method in the first approach or using SCoBeP method in the second approach. The effectiveness of the proposed methods is demonstrated for both synthetic and real video sequences in the experiment section. In addition, the experimental results show that my models are naturally robust in handling super resolution on video sequences affected by scene motions and/or small camera motions. Moreover, in this dissertation, I describe a denoising method using low-rank matrix completion. In the proposed denoising approach, I present a patch-based video denoising algorithm by grouping similar patches and then formulating the problem of removing noise using a decomposition approach for low-rank matrix completion. Experiments show that the proposed approach robustly removes mixed noise such as impulsive noise, Poisson noise, and Gaussian noise from any natural noisy video. Moreover, my approach outperforms state-of-the-art denoising techniques such as VBM3D and 3DWTF in terms of both time and quality. My technique also achieves significant improvement over time against other matrix completion methods

New methods for deep dictionary learning and for image completion

Digital imaging plays an essential role in many aspects of our daily life. However due to the hardware limitations of the imaging devices, the image measurements are usually inpaired and require further processing to enhance the quality of the raw images in order to enable applications on the user side. Image enhancement aims to improve the information content within image measurements by exploiting the properties of the target image and the forward model of the imaging device. In this thesis, we aim to tackle two specific image enhancement problems, that is, single image super-resolution and image completion. First, we present a new Deep Analysis Dictionary Model (DeepAM) which consists of multiple layers of analysis dictionaries with associated soft-thresholding operators and a single layer of synthesis dictionary for single image super-resolution. To achieve an effective deep model, each analysis dictionary has been designed to be composed of an Information Preserving Analysis Dictionary (IPAD) which passes essential information from the input signal to output and a Clustering Analysis Dictionary (CAD) which generates discriminative feature representation. The parameters of the deep analysis dictionary model are optimized using a layer-wise learning strategy. We demonstrate that both the proposed deep dictionary design and the learning algorithm are effective. Simulation results show that the proposed method achieves comparable performance with Deep Neural Networks and other existing methods. We then generalize DeepAM to a Deep Convolutional Analysis Dictionary Model (DeepCAM) by learning convolutional dictionaries instead of unstructured dictionaries. The convolutional dictionary is more suitable for processing high-dimensional signals like images and has only a small number of free parameters. By exploiting the properties of a convolutional dictionary, we present an efficient convolutional analysis dictionary learning algorithm. The IPAD and the CAD parts are learned using variations of the proposed convolutional analysis dictionary learning algorithm. We demonstrate that DeepCAM is an effective multi-layer convolutional model and achieves better performance than DeepAM while using a smaller number of parameters. Finally, we present an image completion algorithm based on dense correspondence between the input image and an exemplar image retrieved from Internet which has been taken at a similar position. The dense correspondence which is estimated using a hierarchical PatchMatch algorithm is usually noisy and with a large occlusion area corresponding to the region to be completed. By modelling the dense correspondence as a smooth field, an Expectation-Maximization (EM) based method is presented to interpolate a smooth field over the occlusion area which is then used to transfer image content from the exemplar image to the input image. Color correction is further applied to diminish the possible color differences between the input image and the exemplar image. Numerical results demonstrate that the proposed image completion algorithm is able to achieve photo realistic image completion results.Open Acces

Gaze-Based Human-Robot Interaction by the Brunswick Model

We present a new paradigm for human-robot interaction based on social signal processing, and in particular on the Brunswick model. Originally, the Brunswick model copes with face-to-face dyadic interaction, assuming that the interactants are communicating through a continuous exchange of non verbal social signals, in addition to the spoken messages. Social signals have to be interpreted, thanks to a proper recognition phase that considers visual and audio information. The Brunswick model allows to quantitatively evaluate the quality of the interaction using statistical tools which measure how effective is the recognition phase. In this paper we cast this theory when one of the interactants is a robot; in this case, the recognition phase performed by the robot and the human have to be revised w.r.t. the original model. The model is applied to Berrick, a recent open-source low-cost robotic head platform, where the gazing is the social signal to be considered

Assessing Variability of EEG and ECG/HRV Time Series Signals Using a Variety of Non-Linear Methods

Time series signals, such as Electroencephalogram (EEG) and Electrocardiogram (ECG) represent the complex dynamic behaviours of biological systems. The analysis of these signals using variety of nonlinear methods is essential for understanding variability within EEG and ECG, which potentially could help unveiling hidden patterns related to underlying physiological mechanisms. EEG is a time varying signal, and electrodes for recording EEG at different positions on the scalp give different time varying signals. There might be correlation between these signals. It is important to know the correlation between EEG signals because it might tell whether or not brain activities from different areas are related. EEG and ECG might be related to each other because both of them are generated from one co-ordinately working body. Investigating this relationship is of interest because it may reveal information about the correlation between EEG and ECG signals. This thesis is about assessing variability of time series data, EEG and ECG, using variety of nonlinear measures. Although other research has looked into the correlation between EEGs using a limited number of electrodes and a limited number of combinations of electrode pairs, no research has investigated the correlation between EEG signals and distance between electrodes. Furthermore, no one has compared the correlation performance for participants with and without medical conditions. In my research, I have filled up these gaps by using a full range of electrodes and all possible combinations of electrode pairs analysed in Time Domain (TD). Cross-Correlation method is calculated on the processed EEG signals for different number unique electrode pairs from each datasets. In order to obtain the distance in centimetres (cm) between electrodes, a measuring tape was used. For most of our participants the head circumference range was 54-58cm, for which a medium-sized I have discovered that the correlation between EEG signals measured through electrodes is linearly dependent on the physical distance (straight-line) distance between them for datasets without medical condition, but not for datasets with medical conditions. Some research has investigated correlation between EEG and Heart Rate Variability (HRV) within limited brain areas and demonstrated the existence of correlation between EEG and HRV. But no research has indicated whether or not the correlation changes with brain area. Although Wavelet Transformations (WT) have been performed on time series data including EEG and HRV signals to extract certain features respectively by other research, so far correlation between WT signals of EEG and HRV has not been analysed. My research covers these gaps by conducting a thorough investigation of all electrodes on the human scalp in Frequency Domain (FD) as well as TD. For the reason of different sample rates of EEG and HRV, two different approaches (named as Method 1 and Method 2) are utilised to segment EEG signals and to calculate Pearson’s Correlation Coefficient for each of the EEG frequencies with each of the HRV frequencies in FD. I have demonstrated that EEG at the front area of the brain has a stronger correlation with HRV than that at the other area in a frequency domain. These findings are independent of both participants and brain hemispheres. Sample Entropy (SE) is used to predict complexity of time series data. Recent research has proposed new calculation methods for SE, aiming to improve the accuracy. To my knowledge, no one has attempted to reduce the computational time of SE calculation. I have developed a new calculation method for time series complexity which could improve computational time significantly in the context of calculating a correlation between EEG and HRV. The results have a parsimonious outcome of SE calculation by exploiting a new method of SE implementation. In addition, it is found that the electrical activity in the frontal lobe of the brain appears to be correlated with the HRV in a time domain. Time series analysis method has been utilised to study complex systems that appear ubiquitous in nature, but limited to certain dynamic systems (e.g. analysing variables affecting stock values). In this thesis, I have also investigated the nature of the dynamic system of HRV. I have disclosed that Embedding Dimension could unveil two variables that determined HRV