34 research outputs found
Matching Pursuits with Random Sequential Subdictionaries
Matching pursuits are a class of greedy algorithms commonly used in signal
processing, for solving the sparse approximation problem. They rely on an atom
selection step that requires the calculation of numerous projections, which can
be computationally costly for large dictionaries and burdens their
competitiveness in coding applications. We propose using a non adaptive random
sequence of subdictionaries in the decomposition process, thus parsing a large
dictionary in a probabilistic fashion with no additional projection cost nor
parameter estimation. A theoretical modeling based on order statistics is
provided, along with experimental evidence showing that the novel algorithm can
be efficiently used on sparse approximation problems. An application to audio
signal compression with multiscale time-frequency dictionaries is presented,
along with a discussion of the complexity and practical implementations.Comment: 20 pages - accepted 2nd April 2012 at Elsevier Signal Processin
Computational Methods for Sparse Solution of Linear Inverse Problems
The goal of the sparse approximation problem is to approximate a target signal using a linear combination of a few elementary signals drawn from a fixed collection. This paper surveys the major practical algorithms for sparse approximation. Specific attention is paid to computational issues, to the circumstances in which individual methods tend to perform well, and to the theoretical guarantees available. Many fundamental questions in electrical engineering, statistics, and applied mathematics can be posed as sparse approximation problems, making these algorithms versatile and relevant to a plethora of applications
Conditioning of Random Block Subdictionaries with Applications to Block-Sparse Recovery and Regression
The linear model, in which a set of observations is assumed to be given by a
linear combination of columns of a matrix, has long been the mainstay of the
statistics and signal processing literature. One particular challenge for
inference under linear models is understanding the conditions on the dictionary
under which reliable inference is possible. This challenge has attracted
renewed attention in recent years since many modern inference problems deal
with the "underdetermined" setting, in which the number of observations is much
smaller than the number of columns in the dictionary. This paper makes several
contributions for this setting when the set of observations is given by a
linear combination of a small number of groups of columns of the dictionary,
termed the "block-sparse" case. First, it specifies conditions on the
dictionary under which most block subdictionaries are well conditioned. This
result is fundamentally different from prior work on block-sparse inference
because (i) it provides conditions that can be explicitly computed in
polynomial time, (ii) the given conditions translate into near-optimal scaling
of the number of columns of the block subdictionaries as a function of the
number of observations for a large class of dictionaries, and (iii) it suggests
that the spectral norm and the quadratic-mean block coherence of the dictionary
(rather than the worst-case coherences) fundamentally limit the scaling of
dimensions of the well-conditioned block subdictionaries. Second, this paper
investigates the problems of block-sparse recovery and block-sparse regression
in underdetermined settings. Near-optimal block-sparse recovery and regression
are possible for certain dictionaries as long as the dictionary satisfies
easily computable conditions and the coefficients describing the linear
combination of groups of columns can be modeled through a mild statistical
prior.Comment: 39 pages, 3 figures. A revised and expanded version of the paper
published in IEEE Transactions on Information Theory (DOI:
10.1109/TIT.2015.2429632); this revision includes corrections in the proofs
of some of the result
Multi-task additive models with shared transfer functions based on dictionary learning
Additive models form a widely popular class of regression models which
represent the relation between covariates and response variables as the sum of
low-dimensional transfer functions. Besides flexibility and accuracy, a key
benefit of these models is their interpretability: the transfer functions
provide visual means for inspecting the models and identifying domain-specific
relations between inputs and outputs. However, in large-scale problems
involving the prediction of many related tasks, learning independently additive
models results in a loss of model interpretability, and can cause overfitting
when training data is scarce. We introduce a novel multi-task learning approach
which provides a corpus of accurate and interpretable additive models for a
large number of related forecasting tasks. Our key idea is to share transfer
functions across models in order to reduce the model complexity and ease the
exploration of the corpus. We establish a connection with sparse dictionary
learning and propose a new efficient fitting algorithm which alternates between
sparse coding and transfer function updates. The former step is solved via an
extension of Orthogonal Matching Pursuit, whose properties are analyzed using a
novel recovery condition which extends existing results in the literature. The
latter step is addressed using a traditional dictionary update rule.
Experiments on real-world data demonstrate that our approach compares favorably
to baseline methods while yielding an interpretable corpus of models, revealing
structure among the individual tasks and being more robust when training data
is scarce. Our framework therefore extends the well-known benefits of additive
models to common regression settings possibly involving thousands of tasks
Motor imagery classification using sparse representation of EEG signals
The human brain is unquestionably the most complex organ of the body as it controls and processes its movement and senses. A healthy brain is able to generate responses to the signals it receives, and transmit messages to the body. Some neural disorders can impair the communication between the brain and the body preventing the transmission of these messages. Brain Computer Interfaces (BCIs) are devices that hold immense potential to assist patients with such disorders by analyzing brain signals, translating and classifying various brain responses, and relaying them to external devices and potentially back to the body. Classifying motor imagery brain signals where the signals are obtained based on imagined movement of the limbs is a major, yet very challenging, step in developing Brain Computer Interfaces (BCIs). Of primary importance is to use less data and computationally efficient algorithms to support real-time BCI. To this end, in this thesis we explore and develop algorithms that exploit the sparse characteristics of EEGs to classify these signals. Different feature vectors are extracted from EEG trials recorded by electrodes placed on the scalp. In this thesis, features from a small spatial region are approximated by a sparse linear combination of few atoms from a multi-class dictionary constructed from the features of the EEG training signals for each class. This is used to classify the signals based on the pattern of their sparse representation using a minimum-residual decision rule. We first attempt to use all the available electrodes to verify the effectiveness of the proposed methods. To support real time BCI, the electrodes are reduced to those near the sensorimotor cortex which are believed to be crucial for motor preparation and imagination. In a second approach, we try to incorporate the effect of spatial correlation across the neighboring electrodes near the sensorimotor cortex. To this end, instead of considering one feature vector at a time, we use a collection of feature vectors simultaneously to find the joint sparse representation of these vectors. Although we were not able to see much improvement with respect to the first approach, we envision that such improvements could be achieved using more refined models that can be subject of future works. The performance of the proposed approaches is evaluated using different features, including wavelet coefficients, energy of the signals in different frequency sub-bands, and also entropy of the signals. The results obtained from real data demonstrate that the combination of energy and entropy features enable efficient classification of motor imagery EEG trials related to hand and foot movements. This underscores the relevance of the energies and their distribution in different frequency sub-bands for classifying movement-specific EEG patterns in agreement with the existence of different levels within the alpha band. The proposed approach is also shown to outperform the state-of-the-art algorithm that uses feature vectors obtained from energies of multiple spatial projections
Sparse image approximation with application to flexible image coding
Natural images are often modeled through piecewise-smooth regions. Region edges, which correspond to the contours of the objects, become, in this model, the main information of the signal. Contours have the property of being smooth functions along the direction of the edge, and irregularities on the perpendicular direction. Modeling edges with the minimum possible number of terms is of key importance for numerous applications, such as image coding, segmentation or denoising. Standard separable basis fail to provide sparse enough representation of contours, due to the fact that this kind of basis do not see the regularity of edges. In order to be able to detect this regularity, a new method based on (possibly redundant) sets of basis functions able to capture the geometry of images is needed. This thesis presents, in a first stage, a study about the features that basis functions should have in order to provide sparse representations of a piecewise-smooth image. This study emphasizes the need for edge-adapted basis functions, capable to accurately capture local orientation and anisotropic scaling of image structures. The need of different anisotropy degrees and orientations in the basis function set leads to the use of redundant dictionaries. However, redundant dictionaries have the inconvenience of giving no unique sparse image decompositions, and from all the possible decompositions of a signal in a redundant dictionary, just the sparsest is needed. There are several algorithms that allow to find sparse decompositions over redundant dictionaries, but most of these algorithms do not always guarantee that the optimal approximation has been recovered. To cope with this problem, a mathematical study about the properties of sparse approximations is performed. From this, a test to check whether a given sparse approximation is the sparsest is provided. The second part of this thesis presents a novel image approximation scheme, based on the use of a redundant dictionary. This scheme allows to have a good approximation of an image with a number of terms much smaller than the dimension of the signal. This novel approximation scheme is based on a dictionary formed by a combination of anisotropically refined and rotated wavelet-like mother functions and Gaussians. An efficient Full Search Matching Pursuit algorithm to perform the image decomposition in such a dictionary is designed. Finally, a geometric image coding scheme based on the image approximated over the anisotropic and rotated dictionary of basis functions is designed. The coding performances of this dictionary are studied. Coefficient quantization appears to be of crucial importance in the design of a Matching Pursuit based coding scheme. Thus, a quantization scheme for the MP coefficients has been designed, based on the theoretical energy upper bound of the MP algorithm and the empirical observations of the coefficient distribution and evolution. Thanks to this quantization, our image coder provides low to medium bit-rate image approximations, while it allows for on the fly resolution switching and several other affine image transformations to be performed directly in the transformed domain
Exploiting Prior Knowledge in Compressed Sensing Wireless ECG Systems
Recent results in telecardiology show that compressed sensing (CS) is a
promising tool to lower energy consumption in wireless body area networks for
electrocardiogram (ECG) monitoring. However, the performance of current
CS-based algorithms, in terms of compression rate and reconstruction quality of
the ECG, still falls short of the performance attained by state-of-the-art
wavelet based algorithms. In this paper, we propose to exploit the structure of
the wavelet representation of the ECG signal to boost the performance of
CS-based methods for compression and reconstruction of ECG signals. More
precisely, we incorporate prior information about the wavelet dependencies
across scales into the reconstruction algorithms and exploit the high fraction
of common support of the wavelet coefficients of consecutive ECG segments.
Experimental results utilizing the MIT-BIH Arrhythmia Database show that
significant performance gains, in terms of compression rate and reconstruction
quality, can be obtained by the proposed algorithms compared to current
CS-based methods.Comment: Accepted for publication at IEEE Journal of Biomedical and Health
Informatic