A unified approach to sparse signal processing

A unified view of the area of sparse signal processing is presented in tutorial form by bringing together various fields in which the property of sparsity has been successfully exploited. For each of these fields, various algorithms and techniques, which have been developed to leverage sparsity, are described succinctly. The common potential benefits of significant reduction in sampling rate and processing manipulations through sparse signal processing are revealed. The key application domains of sparse signal processing are sampling, coding, spectral estimation, array processing, compo-nent analysis, and multipath channel estimation. In terms of the sampling process and reconstruction algorithms, linkages are made with random sampling, compressed sensing and rate of innovation. The redundancy introduced by channel coding i

Overcomplete Dictionary and Deep Learning Approaches to Image and Video Analysis

Extracting useful information while ignoring others (e.g. noise, occlusion, lighting) is an essential and challenging data analyzing step for many computer vision tasks such as facial recognition, scene reconstruction, event detection, image restoration, etc. Data analyzing of those tasks can be formulated as a form of matrix decomposition or factorization to separate useful and/or fill in missing information based on sparsity and/or low-rankness of the data. There has been an increasing number of non-convex approaches including conventional matrix norm optimizing and emerging deep learning models. However, it is hard to optimize the ideal l0-norm or learn the deep models directly and efficiently. Motivated from this challenging process, this thesis proposes two sets of approaches: conventional and deep learning based. For conventional approaches, this thesis proposes a novel online non-convex lp-norm based Robust PCA (OLP-RPCA) approach for matrix decomposition, where 0 < p < 1. OLP-RPCA is developed from the offline version LP-RPCA. A robust face recognition framework is also developed from Robust PCA and sparse coding approaches. More importantly, OLP-RPCA method can achieve real-time performance on large-scale data without parallelizing or implementing on a graphics processing unit. We mathematically and empirically show that our OLP-RPCA algorithm is linear in both the sample dimension and the number of samples. The proposed OLP-RPCA and LP-RPCA approaches are evaluated in various applications including Gaussian/non-Gaussian image denoising, face modeling, real-time background subtraction and video inpainting and compared against numerous state-of-the-art methods to demonstrate the robustness of the algorithms. In addition, this thesis proposes a novel Robust lp-norm Singular Value Decomposition (RP-SVD) method for analyzing two-way functional data. The proposed RP-SVD is formulated as an lp-norm based penalized loss minimization problem. The proposed RP-SVD method is evaluated in four applications, i.e. noise and outlier removal, estimation of missing values, structure from motion reconstruction and facial image reconstruction. For deep learning based approaches, this thesis explores the idea of matrix decomposition via Robust Deep Boltzmann Machines (RDBM), an alternative form of Robust Boltzmann Machines, which aiming at dealing with noise and occlusion for face-related applications, particularly. This thesis proposes an extension to texture modeling in the Deep Appearance Models (DAMs) by using RDBM to enhance its robustness against noise and occlusion. The extended model can cope with occlusion and extreme poses when modeling human faces in 2D image reconstruction. This thesis also introduces new fitting algorithms with occlusion awareness through the mask obtained from the RDBM reconstruction. The proposed approach is evaluated in various applications by using challenging face datasets, i.e. Labeled Face Parts in the Wild (LFPW), Helen, EURECOM and AR databases, to demonstrate its robustness and capabilities

Contribution à la mise en oeuvre de récepteurs et de techniques d'estimation de canal pour les systèmes mobiles de DS-CDMA multi-porteuse

Ce mémoire traite du développement de récepteurs et de techniques déstimation de canal pour les systèmes mobiles sans fil de type DS-CDMA multi-porteuse. Deux problèmes principaux doivent être pris en compte dans ce cas. Premièrement, l'Interférence d'Accès Multiple (IAM) causée par d'autres utilisateurs. Deuxièmement, les propriétés des canaux de propagation dans les systèmes radio mobiles. Ainsi, dans la première partie du manuscrit, nous proposons deux structures adaptatives (dites détection séparée et détection jointe) pour la mise en oeuvre de récepteurs minimisant lérreur quadratique moyenne (MMSE), fondés sur un Algorithme de Projection Affine (APA). Ces récepteurs permettent de supprimer les IAM, notamment lorsque le canal d'évanouissement est invariant dans le temps. Cependant, comme ces récepteurs nécessitent les séquences d'apprentissage de chaque utilisateur actif, nous développons ensuite deux récepteurs adaptatifs dits aveugles, fondés sur un algorithme de type projection affine. Dans ce cas, seule la séquence d'étalement de l'utilisateur désiré est nécessaire. Quand les séquences d'étalement de tous les utilisateurs sont disponibles, un récepteur reposant sur le décorrélateur est aussi proposé et permet d'éliminer les IAM, sans qu'une période pour l'adaptation soit nécessaire. Dans la seconde partie, comme la mise en oeuvre de récepteurs exige léstimation du canal, nous proposons plusieurs algorithmes pour léstimation des canaux d'évanouissement de Rayleigh, variables dans le temps et produits dans les systèmes multi-porteuses. A cette fin, les canaux sont approximés par des processus autorégressifs (AR) d'ordre supérieur à deux. Le premier algorithme repose sur deux filtres de Kalman interactifs pour léstimation conjointe du canal et de ses paramètres AR. Puis, pour nous affranchir des hypothèses de gaussianité nécessaires à la mise en oeuvre d'un filtre optimal de Kalman, nous étudions la pertinence d'une structure fondée sur deux filtres H1 interactifs. Enfin, léstimation de canal peut ^etre vue telle un problème déstimation fondée sur un modèle à erreur- sur-les-variables (EIV). Les paramètres AR du canal et les variances de processus générateur et du bruit d'observation dans la représentation de léspace d'état du système sont dans ce cas estimés conjointement à partir du noyau des matrices d'autocorrélation appropriées.This dissertation deals with the development of receivers and channel estimation techniques for multi-carrier DS- CDMA mobile wireless systems. Two major problems should be taken into account in that case. Firstly, the Multiple Access Interference (MAI) caused by other users. Secondly, the multi-path fading of mobile wireless channels. In the first part of the dissertation, we propose two adaptive structures (called separate and joint detection) to design Minimum Mean Square Error (MMSE) receivers, based on the Affine Projection Algorithm (APA). These receivers are able to suppress the MAI, particularly when the fading channel is time-invariant. However, as they require a training sequence for every active user, we then propose two blind adaptive multiuser receiver structures based on a blind APA-like multiuser detector. In that case, only the knowledge of the spreading code of the desired user is required. When the spreading codes of all users are available, a decorrelating detector based receiver is proposed and is able to completely eliminate the MAI without any training. In the second part, as receiver design usually requires the estimation of the channel, we propose several training-based algorithms for the estimation of time-varying Rayleigh fading channels in multi-carrier systems. For this purpose, the fading channels are approximated by autoregressive (AR) processes whose order is higher than two. The first algorithm makes it possible to jointly estimate the channel and its AR parameters based on two-cross-coupled Kalman filters. Nevertheless, this filtering is based on restrictive Gaussian assumptions. To relax them, we investigate the relevance of a structure based on two-cross-coupled H1 filters. This method consists in minimizing the influence of the disturbances such as the additive noise on the estimation error. Finally, we propose to view the channel estimation as an Errors-In-Variables (EIV) issue. In that case, the channel AR parameters and the variances of both the driving process and the measurement noise in the state-space representation of the system are estimated from the null space of suitable correlation matrices