Manifold Elastic Net: A Unified Framework for Sparse Dimension Reduction

It is difficult to find the optimal sparse solution of a manifold learning based dimensionality reduction algorithm. The lasso or the elastic net penalized manifold learning based dimensionality reduction is not directly a lasso penalized least square problem and thus the least angle regression (LARS) (Efron et al. \cite{LARS}), one of the most popular algorithms in sparse learning, cannot be applied. Therefore, most current approaches take indirect ways or have strict settings, which can be inconvenient for applications. In this paper, we proposed the manifold elastic net or MEN for short. MEN incorporates the merits of both the manifold learning based dimensionality reduction and the sparse learning based dimensionality reduction. By using a series of equivalent transformations, we show MEN is equivalent to the lasso penalized least square problem and thus LARS is adopted to obtain the optimal sparse solution of MEN. In particular, MEN has the following advantages for subsequent classification: 1) the local geometry of samples is well preserved for low dimensional data representation, 2) both the margin maximization and the classification error minimization are considered for sparse projection calculation, 3) the projection matrix of MEN improves the parsimony in computation, 4) the elastic net penalty reduces the over-fitting problem, and 5) the projection matrix of MEN can be interpreted psychologically and physiologically. Experimental evidence on face recognition over various popular datasets suggests that MEN is superior to top level dimensionality reduction algorithms.Comment: 33 pages, 12 figure

Universal and efficient compressed sensing by spread spectrum and application to realistic Fourier imaging techniques

We advocate a compressed sensing strategy that consists of multiplying the signal of interest by a wide bandwidth modulation before projection onto randomly selected vectors of an orthonormal basis. Firstly, in a digital setting with random modulation, considering a whole class of sensing bases including the Fourier basis, we prove that the technique is universal in the sense that the required number of measurements for accurate recovery is optimal and independent of the sparsity basis. This universality stems from a drastic decrease of coherence between the sparsity and the sensing bases, which for a Fourier sensing basis relates to a spread of the original signal spectrum by the modulation (hence the name "spread spectrum"). The approach is also efficient as sensing matrices with fast matrix multiplication algorithms can be used, in particular in the case of Fourier measurements. Secondly, these results are confirmed by a numerical analysis of the phase transition of the l1- minimization problem. Finally, we show that the spread spectrum technique remains effective in an analog setting with chirp modulation for application to realistic Fourier imaging. We illustrate these findings in the context of radio interferometry and magnetic resonance imaging.Comment: Submitted for publication in EURASIP Journal on Advances in Signal Processin

Iterative algorithms for total variation-like reconstructions in seismic tomography

A qualitative comparison of total variation like penalties (total variation, Huber variant of total variation, total generalized variation, ...) is made in the context of global seismic tomography. Both penalized and constrained formulations of seismic recovery problems are treated. A number of simple iterative recovery algorithms applicable to these problems are described. The convergence speed of these algorithms is compared numerically in this setting. For the constrained formulation a new algorithm is proposed and its convergence is proven.Comment: 28 pages, 8 figures. Corrected sign errors in formula (25

Compressive Fluorescence Microscopy for Biological and Hyperspectral Imaging

The mathematical theory of compressed sensing (CS) asserts that one can acquire signals from measurements whose rate is much lower than the total bandwidth. Whereas the CS theory is now well developed, challenges concerning hardware implementations of CS-based acquisition devices---especially in optics---have only started being addressed. This paper presents an implementation of compressive sensing in fluorescence microscopy and its applications to biomedical imaging. Our CS microscope combines a dynamic structured wide-field illumination and a fast and sensitive single-point fluorescence detection to enable reconstructions of images of fluorescent beads, cells and tissues with undersampling ratios (between the number of pixels and number of measurements) up to 32. We further demonstrate a hyperspectral mode and record images with 128 spectral channels and undersampling ratios up to 64, illustrating the potential benefits of CS acquisition for higher dimensional signals which typically exhibits extreme redundancy. Altogether, our results emphasize the interest of CS schemes for acquisition at a significantly reduced rate and point out to some remaining challenges for CS fluorescence microscopy.Comment: Submitted to Proceedings of the National Academy of Sciences of the United States of Americ

On visualizing continuous turbulence scales

Turbulent flows are multi‐scale with vortices spanning a wide range of scales continuously. Due to such complexities, turbulence scales are particularly difficult to analyse and visualize. In this work, we present a novel and efficient optimization‐based method for continuous‐scale turbulence structure visualization with scale decomposition directly in the Kolmogorov energy spectrum. To achieve this, we first derive a new analytical objective function based on integration approximation. Using this new formulation, we can significantly improve the efficiency of the underlying optimization process and obtain the desired filter in the Kolmogorov energy spectrum for scale decomposition. More importantly, such a decomposition allows a ‘continuous‐scale visualization’ that enables us to efficiently explore the decomposed turbulence scales and further analyse the turbulence structures in a continuous manner. With our approach, we can present scale visualizations of direct numerical simulation data sets continuously over the scale domain for both isotropic and boundary layer turbulent flows. Compared with previous works on multi‐scale turbulence analysis and visualization, our method is highly flexible and efficient in generating scale decomposition and visualization results. The application of the proposed technique to both isotropic and boundary layer turbulence data sets verifies the capability of our technique to produce desirable scale visualization results

Atoms of all channels, unite! Average case analysis of multi-channel sparse recovery using greedy algorithms

This paper provides new results on computing simultaneous sparse approximations of multichannel signals over redundant dictionaries using two greedy algorithms. The first one, p-thresholding, selects the S atoms that have the largest $p$-correlation while the second one, p-simultaneous matching pursuit (p-SOMP), is a generalisation of an algorithm studied by Tropp. We first provide exact recovery conditions as well as worst case analyses of all algorithms. The results, expressed using the standard cumulative coherence, are very reminiscent of the single channel case and, in particular, impose stringent restrictions on the dictionary. We unlock the situation by performing an average case analysis of both algorithms. First, we set up a general probabilistic signal model in which the coefficients of the atoms are drawn at random from the standard gaussian distribution. Second, we show that under this model, and with mild conditions on the coherence, the probability that p-thresholding and p-SOMP fail to recover the correct components is overwhelmingly small and gets smaller as the number of channels increases. Furthermore, we analyse the influence of selecting the set of correct atoms at random. We show that, if the dictionary satisfies a uniform uncertainty principle, the probability that simultaneous OMP fails to recover any sufficiently sparse set of atoms gets increasingly smaller as the number of channels increases

Compressed Sensing of Sparse Multipath MIMO Channels with Superimposed Training Sequence

Recent advances in multiple-input multiple-output (MIMO) systems have renewed the interests of researchers to further explore this area for addressing various dynamic challenges of emerging radio communication networks. Various measurement campaigns reported recently in the literature show that physical multipath MIMO channels exhibit sparse impulse response structure in various outdoor radio propagation environments. Therefore, a comprehensive physical description of sparse multipath MIMO channels is presented in first part of this paper. Superimposing a training sequence (low power, periodic) over the information sequence offers an improvement in the spectral efficiency by avoiding the use of dedicated time/frequency slots for the training sequence, which is unlike the traditional schemes. The main contribution of this paper includes three superimposed training (SiT) sequence based channel estimation techniques for sparse multipath MIMO channels. The proposed techniques exploit the compressed sensing theory and prior available knowledge of channel’s sparsity. The proposed sparse MIMO channel estimation techniques are named as, SiT based compressed channel sensing (SiT-CCS), SiT based hardlimit thresholding with CCS (SiT-ThCCS), and SiT training based match pursuit (SiT-MP). Bit error rate (BER) and normalized channel mean square error are used as metrics for the simulation analysis to gauge the performance of proposed techniques. A comparison of the proposed schemes with a notable first order statistics based SiT least squares (SiT-LS) estimation technique is presented to establish the improvements achieved by the proposed schemes. For sparse multipath time-invariant MIMO communication channels, it is observed that SiT-CCS, SiT-MP, and SiT-ThCCS can provide an improvement up to 2, 3.5, and 5.2 dB in the MSE at signal to noise ratio (SNR) of 12 dB when compared to SiT-LS, respectively. Moreover, for BER=10 −1.9 BER=10−1.9, the proposed SiT-CCS, SiT-MP, and SiT-ThCCS, compared to SiT-LS, can offer a gain of about 1, 2.5, and 3.5 dB in the SNR, respectively. The performance gain in MSE and BER is observed to improve with an increase in the channel sparsity