Multichannel sampling of finite rate of innovation signals

Recently there has been a surge of interest in sampling theory in signal processing community. New efficient sampling techniques have been developed that allow sampling and perfectly reconstructing some classes of non-bandlimited signals at sub-Nyquist rates. Depending on the setup used and reconstruction method involved, these schemes go under different names such as compressed sensing (CS), compressive sampling or sampling signals with finite rate of innovation (FRI). In this thesis we focus on the theory of sampling non-bandlimited signals with parametric structure or specifically signals with finite rate of innovation. Most of the theory on sampling FRI signals is based on a single acquisition device with one-dimensional (1-D) signals. In this thesis, we extend these results to the case of 2-D signals and multichannel acquisition systems. The essential issue in multichannel systems is that while each channel receives the input signal, it may introduce different unknown delays, gains or affine transformations which need to be estimated from the samples together with the signal itself. We pose both the calibration of the channels and the signal reconstruction stage as a parametric estimation problem and demonstrate that a simultaneous exact synchronization of the channels and reconstruction of the FRI signal is possible. Furthermore, because in practice perfect noise-free channels do not exist, we consider the case of noisy measurements and show that by considering Cramer-Rao bounds as well as numerical simulations, the multichannel systems are more resilient to noise than the single-channel ones. Finally, we consider the problem of system identification based on the multichannel and finite rate of innovation sampling techniques. First, by employing our multichannel sampling setup, we propose a novel algorithm for system identification problem with known input signal, that is for the case when both the input signal and the samples are known. Then we consider the problem of blind system identification and propose a novel algorithm for simultaneously estimating the input FRI signal and also the unknown system using an iterative algorithm

Distributed sampling and compression of scenes with finite rate of innovation in camera sensor networks

We study the problem of distributed sampling and compression in sensor networks when the sensors are digital cameras that acquire a 3-D visual scene of interest from different viewing positions. We assume that sensors cannot communicate among themselves, but can process their acquired data and transmit it to a common central receiver. The main task of the receiver is then to reconstruct the best possible estimation of the original scene and the natural issue, in this context, is to understand the interplay in the reconstruction between sampling and distributed compression. In this paper, we show that if the observed scene belongs to the class of signals that can be represented with a finite number of parameters, we can determine the minimum number of sensors that allows perfect reconstruction of the scene. Then, we present a practical distributed coding approach that leads to a rate-distortion behaviour at the decoder that is independent of the number of sensors, when this number increases beyond the critical sampling. In other words, we show that the distortion at the decoder does not depend on the number of sensors used, but only on the total number of bits that can be transmitted from the sensors to the receiver.

Coherent multi-dimensional segmentation of multiview images using a variational framework and applications to image based rendering

Image Based Rendering (IBR) and in particular light field rendering has attracted a lot of attention for interpolating new viewpoints from a set of multiview images. New images of a scene are interpolated directly from nearby available ones, thus enabling a photorealistic rendering. Sampling theory for light fields has shown that exact geometric information in the scene is often unnecessary for rendering new views. Indeed, the band of the function is approximately limited and new views can be rendered using classical interpolation methods. However, IBR using undersampled light fields suffers from aliasing effects and is difficult particularly when the scene has large depth variations and occlusions. In order to deal with these cases, we study two approaches: New sampling schemes have recently emerged that are able to perfectly reconstruct certain classes of parametric signals that are not bandlimited but characterized by a finite number of parameters. In this context, we derive novel sampling schemes for piecewise sinusoidal and polynomial signals. In particular, we show that a piecewise sinusoidal signal with arbitrarily high frequencies can be exactly recovered given certain conditions. These results are applied to parametric multiview data that are not bandlimited. We also focus on the problem of extracting regions (or layers) in multiview images that can be individually rendered free of aliasing. The problem is posed in a multidimensional variational framework using region competition. In extension to previous methods, layers are considered as multi-dimensional hypervolumes. Therefore the segmentation is done jointly over all the images and coherence is imposed throughout the data. However, instead of propagating active hypersurfaces, we derive a semi-parametric methodology that takes into account the constraints imposed by the camera setup and the occlusion ordering. The resulting framework is a global multi-dimensional region competition that is consistent in all the images and efficiently handles occlusions. We show the validity of the approach with captured light fields. Other special effects such as augmented reality and disocclusion of hidden objects are also demonstrated

Feature Extraction for image super-resolution using finite rate of innovation principles

To understand a real-world scene from several multiview pictures, it is necessary to find the disparities existing between each pair of images so that they are correctly related to one another. This process, called image registration, requires the extraction of some specific information about the scene. This is achieved by taking features out of the acquired images. Thus, the quality of the registration depends largely on the accuracy of the extracted features. Feature extraction can be formulated as a sampling problem for which perfect re- construction of the desired features is wanted. The recent sampling theory for signals with finite rate of innovation (FRI) and the B-spline theory offer an appropriate new frame- work for the extraction of features in real images. This thesis first focuses on extending the sampling theory for FRI signals to a multichannel case and then presents exact sampling results for two different types of image features used for registration: moments and edges. In the first part, it is shown that the geometric moments of an observed scene can be retrieved exactly from sampled images and used as global features for registration. The second part describes how edges can also be retrieved perfectly from sampled images for registration purposes. The proposed feature extraction schemes therefore allow in theory the exact registration of images. Indeed, various simulations show that the proposed extraction/registration methods overcome traditional ones, especially at low-resolution. These characteristics make such feature extraction techniques very appropriate for applications like image super-resolution for which a very precise registration is needed. The quality of the super-resolved images obtained using the proposed feature extraction meth- ods is improved by comparison with other approaches. Finally, the notion of polyphase components is used to adapt the image acquisition model to the characteristics of real digital cameras in order to run super-resolution experiments on real images

Feature Extraction for Image Super-resolution using Finite Rate of Innovation Principles

To understand a real-world scene from several multiview pictures, it is necessary to find the disparities existing between each pair of images so that they are correctly related to one another., This process. called image registration, reguires the extraction of some specific information about the scene. This is achieved by taking features out of the acquired imaqes. Thus, the quality of the, registration depends largely on the accuracy of the extracted features. Feature extraction can be formulated as a sampling problem for which perfect reconstruction of the, desired features is wanted. The recent sampling theory for signals with finite rate of innovation (FR/), and the B-spline theory offer an appropriate new framework for the extraction of features in real, images. This thesis first focuses on extending the sampling theory for FRI signals to a multichannel, case and then presents exact sampling results for two different types of image features used for, registration: moments and edges. In the first part, it is shown that the geometric moments of an observed scene can be retrieved exactly from sampled images and used as global features for registration. The second part describes how edges can also be retrieved perfectly from sampled images for registration purposes. The proposed feature extraction schemes therefore allow in theory the exact registration of images. Indeed, various simulations show that the proposed extraction/registration methods overcome traditional ones, especially at low-resolution. These characteristics make such feature extraction techniques very appropriate for applications like image super-resolution for which a very precise registration is needed. The quality of the superresolved images obtained using the proposed feature extraction methods is improved by comparison with other approaches. Finally, the notion of polyphase components is used to adapt the imaqe acquisition model to the characteristics of real digital cameras in order to run super-resolution experiments on real images