Sampling the Multiple Facets of Light

The theme of this thesis revolves around three important manifestations of light, namely its corpuscular, wave and electromagnetic nature. Our goal is to exploit these principles to analyze, design and build imaging modalities by developing new signal processing and algorithmic tools, based in particular on sampling and sparsity concepts. First, we introduce a new sampling scheme called variable pulse width, which is based on the finite rate of innovation (FRI) sampling paradigm. This new framework enables to sample and perfectly reconstruct weighted sums of Lorentzians; perfect reconstruction from sampled signals is guaranteed by a set of theorems. Second, we turn to the context of light and study its reflection, which is based on the corpuscular model of light. More precisely, we propose to use our FRI-based model to represent bidirectional reflectance distribution functions. We develop dedicated light domes to acquire reflectance functions and use the measurements obtained to demonstrate the usefulness and versatility of our model. In particular, we concentrate on the representation of specularities, which are sharp and bright components generated by the direct reflection of light on surfaces. Third, we explore the wave nature of light through Lippmann photography, a century-old photography technique that acquires the entire spectrum of visible light. This fascinating process captures interferences patterns created by the exposed scene inside the depth of a photosensitive plate. By illuminating the developed plate with a neutral light source, the reflected spectrum corresponds to that of the exposed scene. We propose a mathematical model which precisely explains the technique and demonstrate that the spectrum reproduction suffers from a number of distortions due to the finite depth of the plate and the choice of reflector. In addition to describing these artifacts, we describe an algorithm to invert them, essentially recovering the original spectrum of the exposed scene. Next, the wave nature of light is further generalized to the electromagnetic theory, which we invoke to leverage the concept of polarization of light. We also return to the topic of the representation of reflectance functions and focus this time on the separation of the specular component from the other reflections. We exploit the fact that the polarization of light is preserved in specular reflections and investigate camera designs with polarizing micro-filters with different orientations placed just in front of the camera sensor; the different polarizations of the filters create a mosaic image, from which we propose to extract the specular component. We apply our demosaicing method to several scenes and additionally demonstrate that our approach improves photometric stereo. Finally, we delve into the problem of retrieving the phase information of a sparse signal from the magnitude of its Fourier transform. We propose an algorithm that resolves the phase retrieval problem for sparse signals in three stages. Unlike traditional approaches that recover a discrete approximation of the underlying signal, our algorithm estimates the signal on a continuous domain, which makes it the first of its kind. The concluding chapter outlines several avenues for future research, like new optical devices such as displays and digital cameras, inspired by the topic of Lippmann photography

Sensing ECG signals with variable pulse width finite rate of innovation

Mobile health is gradually taking more importance in our society and the need of new power efficient devices acquiring biosignals for long periods of time is becoming substantial. In this thesis, we study the power reduction we could achieve on ECG sensing devices. Emphasis is made on reducing the number of samples both during the sensing phase and the compression phase. To that end, a new scheme called variable pulse width finite rate of innovation (VPW-FRI) is investigated. This new technique relies on the classical finite rate of innovation (FRI) theory and enables the use of a sum of asymmetric Cauchy-based pulses to model ECG signals. Research is done in order to implement VPW in practice and its performance are carefully analysed. Among others, we consider the potential instability of the method, we study its compression effectiveness and compare it with compression schemes widespread in the literature. We also evaluate the spectrum extrapolation performance of VPW when fed with signals sampled at sub-Nyquist rates and propose a modification that improves it. Furthermore, we introduce a method based on the similarities between different heart beats that reduces the computational costs of VPW. The parametric nature of VPW finally allows us to use it as a noise reduction algorithm. In parallel, we review and test a non-uniform sensing technique that adapts the sampling rate to the slope of the signal

Lippmann Photography: A Signal Processing Perspective

Lippmann (or interferential) photography is the first and only analog photography method that can capture the full color spectrum of a scene in a single take. This technique, invented more than a hundred years ago, records the colors by creating interference patterns inside the photosensitive plate. Lippmann photography provides a great opportunity to demonstrate several fundamental concepts in signal processing. Conversely, a signal processing perspective enables us to shed new light on the technique. In our previous work, we analyzed the spectra of historical Lippmann plates using our own mathematical model. In this paper, we provide the derivation of this model and validate it experimentally. We highlight new behaviors whose explanations were ignored by physicists to date. In particular, we show that the spectra generated by Lippmann plates are in fact distorted versions of the original spectra. We also show that these distortions are influenced by the thickness of the plate and the reflection coefficient of the reflective medium used in the capture of the photographs. We verify our model with extensive experiments on our own Lippmann photographs.Comment: 12 pages, 18 figures, to be published in Transactions in Signal Processin

Accurate recovery of a specularity from a few samples of the reflectance function

We present a new technique for estimating the specular peak of the bidirectional reflectance distribution function (BRDF) based on finite rate of innovation (FRI) sampling. The specular component of the BRDF varies rapidly, so it is challenging to acquire it by pointwise sampling. Yet, the knowledge of its precise location is key to render realistically complex materials. We show how to adapt the FRI framework to accurately determine the location of a single pulse when the sampling kernel is unknown. We use this result to determine the position of the specularity, and then estimate its shape by non-linear optimization. We demonstrate the feasibility of our approach in simulations and via a practical experiment using a custom-built BRDF acquisition device

Structure from sound with incomplete data

In this paper, we consider the problem of jointly localizing a microphone array and identifying the direction of arrival of acoustic events. Under the assumption that the sources are in the far field, this problem can be formulated as a constrained low-rank matrix factorization with an unknown column offset. Our focus is on handling missing entries, particularly when the measurement matrix does not contain a single complete column. This case has not received attention in the literature and is not handled by existing algorithms, however it is prevalent in practice. We propose an iterative algorithm that works with pairwise differences between the measurements eliminating the dependence on the unknown offset. We demonstrate state-of-the-art performance both in terms of accuracy and versatility

Finite Rate of Innovation Based Modeling and Compression of ECG Signals

Mobile health is gaining increasing importance for society and the quest for new power efficient devices sampling biosignals is becoming critical. We discuss a new scheme called Variable PulseWidth Finite Rate of Innovation (VPW-FRI) to model and compress ECG signals. This technique generalizes classical FRI estimation to enable the use of a sum of asymmetric Cauchy-based pulses for modeling electrocardiogram (ECG) signals. We experimentally show that VPW-FRI indeed models ECG signals with increased accuracy compared to current standards. In addition, we study the compression efficiency of the method: compared with various widely used compression schemes, we showcase improvements in terms of compression efficiency as well as sampling rate

Sampling and Exact Reconstruction of Pulses with Variable Width

Recent sampling results enable the reconstruction of signals composed of streams of fixed-shaped pulses. These results have found applications in topics as varied as channel estimation, biomedical imaging and radio astronomy. However, in many real signals, the pulse shapes vary throughout the signal. In this paper, we show how to sample and perfectly reconstruct Lorentzian pulses with variable width. Since a stream of Lorentzian pulses has a finite number of degrees of freedom per unit time, it belongs to the class of signals with finite rate of innovation (FRI). In the noiseless case, perfect recovery is guaranteed by a set of theorems. In addition, we verify that our algorithm is robust to model-mismatch and noise. This allows us to apply the technique to two practical applications: electrocardiogram (ECG) compression and bidirectional reflectance distribution function (BRDF) sampling. ECG signals are one dimensional, but the BRDF is a higher dimensional signal, which is more naturally expressed in a spherical coordinate system; this motivated us to extend the theory to the 2D and spherical cases. Experiments on real data demonstrate the viability of the proposed model for ECG acquisition and compression, as well as the efficient representation and low-rate sampling of specular BRDFs

Multichannel ECG Analysis Using VPW-FRI

In this paper, we present an application of Variable Pulse Width Finite Rate of Innovation (VPW-FRI) in dealing with multichannel Electrocardiogram (ECG) data using a common annihilator. By extending the conventional FRI model to include additional parameters such as pulse width and asymmetry, VPWFRI has been able to deal with a more general class of pulses. The common annihilator, which is introduced in the annihilating filter step, shows a common support in multichannel ECG data, which provides interesting possibilities in compression. A model based de-noising method will be presented which is fast and noniterative. Also, an application to detect QRS complexes in ECG signals will be demonstrated. The results will show the robustness of the common annihilator and the QRS detection even in the presence of noise

Super Resolution Phase Retrieval for Sparse Signals

In a variety of fields, in particular those involving imaging and optics, we often measure signals whose phase is missing or has been irremediably distorted. Phase retrieval attempts to recover the phase information of a signal from the magnitude of its Fourier transform to enable the reconstruction of the original signal. Solving the phase retrieval problem is equivalent to recovering a signal from its auto-correlation function. In this paper, we assume the original signal to be sparse; this is a natural assumption in many applications, such as X-ray crystallography, speckle imaging and blind channel estimation. We propose an algorithm that resolves the phase retrieval problem in three stages: i) we leverage the finite rate of innovation sampling theory to super-resolve the auto-correlation function from a limited number of samples, ii) we design a greedy algorithm that identifies the locations of a sparse solution given the super-resolved auto-correlation function, iii) we recover the amplitudes of the atoms given their locations and the measured auto-correlation function. Unlike traditional approaches that recover a discrete approximation of the underlying signal, our algorithm estimates the signal on a continuous domain, which makes it the first of its kind. Along with the algorithm, we derive its performance bound with a theoretical analysis and propose a set of enhancements to improve its computational complexity and noise resilience. Finally, we demonstrate the benefits of the proposed method via a comparison against Charge Flipping, a notable algorithm in crystallography

Extension of FRI for modeling of electrocardiogram signals

Recent work has developed a modeling method applicable to certain types of signals having a "finite rate of innovation" (FRI). Such signals contain a sparse collection of time- or frequency-limited pulses having a restricted set of allowable pulse shapes. A limitation of past work on FRI is that all of the pulses must have the same shape. Many real signals, including electrocardiograms, consist of pulses with varying widths and asymmetry, and therefore are not well fit by the past FRI methods. We present an extension of FRI allowing pulses having variable pulse width (VPW) and asymmetry. We show example results for electrocardiograms and discuss the possibility of application to signal compression and diagnostics