A Method for Finding Structured Sparse Solutions to Non-negative Least Squares Problems with Applications

Demixing problems in many areas such as hyperspectral imaging and differential optical absorption spectroscopy (DOAS) often require finding sparse nonnegative linear combinations of dictionary elements that match observed data. We show how aspects of these problems, such as misalignment of DOAS references and uncertainty in hyperspectral endmembers, can be modeled by expanding the dictionary with grouped elements and imposing a structured sparsity assumption that the combinations within each group should be sparse or even 1-sparse. If the dictionary is highly coherent, it is difficult to obtain good solutions using convex or greedy methods, such as non-negative least squares (NNLS) or orthogonal matching pursuit. We use penalties related to the Hoyer measure, which is the ratio of the $l_1$ and $l_2$ norms, as sparsity penalties to be added to the objective in NNLS-type models. For solving the resulting nonconvex models, we propose a scaled gradient projection algorithm that requires solving a sequence of strongly convex quadratic programs. We discuss its close connections to convex splitting methods and difference of convex programming. We also present promising numerical results for example DOAS analysis and hyperspectral demixing problems.Comment: 38 pages, 14 figure

Sparse representations of signals for information recovery from incomplete data

Mathematical modeling of inverse problems in imaging, such as inpainting, deblurring and denoising, results in ill-posed, i.e. underdetermined linearsystems. Sparseness constraintis used often to regularize these problems.That is because many classes of discrete signals (e.g. naturalimages), when expressed as vectors in a high-dimensional space, are sparse in some predefined basis or a frame(fixed or learned). An efficient approach to basis / frame learning is formulated using the independent component analysis (ICA)and biologically inspired linear model of sparse coding. In the learned basis, the inverse problem of data recovery and removal of impulsive noise is reduced to solving sparseness constrained underdetermined linear system of equations. The same situation occurs in bioinformatics data analysis when novel type of linear mixture model with a reference sample is employed for feature extraction. Extracted features can be used for disease prediction and biomarker identification

Group-structured and independent subspace based dictionary learning

Thanks to the several successful applications, sparse signal representation has become one of the most actively studied research areas in mathematics. However, in the traditional sparse coding problem the dictionary used for representation is assumed to be known. In spite of the popularity of sparsity and its recently emerged structured sparse extension, interestingly, very few works focused on the learning problem of dictionaries to these codes. In the first part of the paper, we develop a dictionary learning method which is (i) online, (ii) enables overlapping group structures with (iii) non-convex sparsity-inducing regularization and (iv) handles the partially observable case. To the best of our knowledge, current methods can exhibit two of these four desirable properties at most. We also investigate several interesting special cases of our framework and demonstrate its applicability in inpainting of natural signals, structured sparse non-negative matrix factorization of faces and collaborative filtering. Complementing the sparse direction we formulate a novel component-wise acting, epsilon-sparse coding scheme in reproducing kernel Hilbert spaces and show its equivalence to a generalized class of support vector machines. Moreover, we embed support vector machines to multilayer perceptrons and show that for this novel kernel based approximation approach the backpropagation procedure of multilayer perceptrons can be generalized. In the second part of the paper, we focus on dictionary learning making use of independent subspace assumption instead of structured sparsity. The corresponding problem is called independent subspace analysis (ISA), or independent component analysis (ICA) if all the hidden, independent sources are one-dimensional. One of the most fundamental results of this research field is the ISA separation principle, which states that the ISA problem can be solved by traditional ICA up to permutation. This principle (i) forms the basis of the state-of-the-art ISA solvers and (ii) enables one to estimate the unknown number and the dimensions of the sources efficiently. We (i) extend the ISA problem to several new directions including the controlled, the partially observed, the complex valued and the nonparametric case and (ii) derive separation principle based solution techniques for the generalizations. This solution approach (i) makes it possible to apply state-of-the-art algorithms for the obtained subproblems (in the ISA example ICA and clustering) and (ii) handles the case of unknown dimensional sources. Our extensive numerical experiments demonstrate the robustness and efficiency of our approach

Compressed sensing on terahertz imaging

Most terahertz (THz) time-domain (pulsed) imaging experiments that have been performed by raster scanning the object relative to a focused THz beam require minutes or even hours to acquire a complete image. This slow image acquisition is a major limiting factor for real-time applications. Other systems using focal plane detector arrays can acquire images in real-time, but they are too expensive or are limited by low sensitivity in the THz range. More importantly, such systems cannot provide spectroscopic information of the sample. To develop faster and more efficient THz time-domain (pulsed) imaging systems, this research used random projection approach to reconstruct THz images from the synthetic and real-world THz data based on the concept of compressed/compressive sensing/sampling (CS). Compared with conventional THz time-domain (pulsed) imaging, no raster scanning of the object is required. The simulation results demonstrated that CS has great potential for real-time THz imaging systems because its use can dramatically reduce the number of measurements in such systems. We then implemented two different CS-THz systems based on the random projection method. One is a compressive THz time-domain (pulsed) spectroscopic imaging system using a set of independent optimized masks. A single-point THz detector, together with a set of 40 optimized two-dimensional binary masks, was used to measure the THz waveforms transmitted through a sample. THz time- and frequency-domain images of the sample comprising 20×20 pixels were subsequently reconstructed. This demonstrated that both the spatial distribution and the spectral characteristics of a sample can be obtained by this means. Compared with conventional THz time-domain (pulsed) imaging, ten times fewer THz spectra need to be taken. In order to further speed up the image acquisition and reconstruction process, another hardware implementation - a single rotating mask (i.e., the spinning disk) with random binary patterns - was utilized to spatially modulate a collimated THz. After propagating through the sample, the THz beam was measured using a single detector, and a THz image was subsequently reconstructed using the CS approach. This demonstrated that a 32×32 pixel image could be obtained from 160 to 240 measurements. This spinning disk configuration allows the use of an electric motor to rotate the spinning disk, thus enabling the experiment to be performed automatically and continuously. To the best of our knowledge, this is the first experimental implementation of a spinning disk configuration for high speed compressive image acquisition. A three-dimensional (3D) joint reconstruction approach was developed to reconstruct THz images from random/incomplete subsets of THz data. Such a random sampling method provides a fast THz imaging acquisition and also simplifies the current THz imaging hardware implementation. The core idea is extended in image inpainting to the case of 3D data. Our main objective is to exploit both spatial and spectral/temporal information for recovering the missing samples. It has been shown that this approach has superiority over the case where the spectral/temporal images are treated independently. We first proposed to learn a spatio-spectral/temporal dictionary from a subset of available training data. Using this dictionary, the THz images can then be jointly recovered from an incomplete set of observations. The simulation results using the measured THz image data confirm that this 3D joint reconstruction approach also provides a significant improvement over the existing THz imaging methods

MS FT-2-2 7 Orthogonal polynomials and quadrature: Theory, computation, and applications

Quadrature rules find many applications in science and engineering. Their analysis is a classical area of applied mathematics and continues to attract considerable attention. This seminar brings together speakers with expertise in a large variety of quadrature rules. It is the aim of the seminar to provide an overview of recent developments in the analysis of quadrature rules. The computation of error estimates and novel applications also are described

Generalized averaged Gaussian quadrature and applications

A simple numerical method for constructing the optimal generalized averaged Gaussian quadrature formulas will be presented. These formulas exist in many cases in which real positive GaussKronrod formulas do not exist, and can be used as an adequate alternative in order to estimate the error of a Gaussian rule. We also investigate the conditions under which the optimal averaged Gaussian quadrature formulas and their truncated variants are internal