Darth Fader: Using wavelets to obtain accurate redshifts of spectra at very low signal-to-noise

We present the DARTH FADER algorithm, a new wavelet-based method for estimating redshifts of galaxy spectra in spectral surveys that is particularly adept in the very low SNR regime. We use a standard cross-correlation method to estimate the redshifts of galaxies, using a template set built using a PCA analysis on a set of simulated, noise-free spectra. Darth Fader employs wavelet filtering to both estimate the continuum & to extract prominent line features in each galaxy spectrum. A simple selection criterion based on the number of features present in the spectrum is then used to clean the catalogue: galaxies with fewer than six total features are removed as we are unlikely to obtain a reliable redshift estimate. Applying our wavelet-based cleaning algorithm to a simulated testing set, we successfully build a clean catalogue including extremely low signal-to-noise data (SNR=2.0), for which we are able to obtain a 5.1% catastrophic failure rate in the redshift estimates (compared with 34.5% prior to cleaning). We also show that for a catalogue with uniformly mixed SNRs between 1.0 & 20.0, with realistic pixel-dependent noise, it is possible to obtain redshifts with a catastrophic failure rate of 3.3% after cleaning (as compared to 22.7% before cleaning). Whilst we do not test this algorithm exhaustively on real data, we present a proof of concept of the applicability of this method to real data, showing that the wavelet filtering techniques perform well when applied to some typical spectra from the SDSS archive. The Darth Fader algorithm provides a robust method for extracting spectral features from very noisy spectra. The resulting clean catalogue gives an extremely low rate of catastrophic failures, even when the spectra have a very low SNR. For very large sky surveys, this technique may offer a significant boost in the number of faint galaxies with accurately determined redshifts.Comment: 22 pages, 15 figures. Accepted for publication in Astronomy & Astrophysic

Objective and efficient terahertz signal denoising by transfer function reconstruction

As an essential processing step in many disciplines, signal denoising efficiently improves data quality without extra cost. However, it is relatively under-utilized for terahertz spectroscopy. The major technique reported uses wavelet denoising in the time-domain, which has a fuzzy physical meaning and limited performance in low-frequency and water-vapor regions. Here, we work from a new perspective by reconstructing the transfer function to remove noise-induced oscillations. The method is fully objective without a need for defining a threshold. Both reflection imaging and transmission imaging were conducted. The experimental results show that both low- and high-frequency noise and the water-vapor influence were efficiently removed. The spectrum accuracy was also improved, and the image contrast was significantly enhanced. The signal-to-noise ratio of the leaf image was increased up to 10 dB, with the 6 dB bandwidth being extended by over 0.5 THz

Image Decomposition and Separation Using Sparse Representations: An Overview

This paper gives essential insights into the use of sparsity and morphological diversity in image decomposition and source separation by reviewing our recent work in this field. The idea to morphologically decompose a signal into its building blocks is an important problem in signal processing and has far-reaching applications in science and technology. Starck , proposed a novel decomposition method—morphological component analysis (MCA)—based on sparse representation of signals. MCA assumes that each (monochannel) signal is the linear mixture of several layers, the so-called morphological components, that are morphologically distinct, e.g., sines and bumps. The success of this method relies on two tenets: sparsity and morphological diversity. That is, each morphological component is sparsely represented in a specific transform domain, and the latter is highly inefficient in representing the other content in the mixture. Once such transforms are identified, MCA is an iterative thresholding algorithm that is capable of decoupling the signal content. Sparsity and morphological diversity have also been used as a novel and effective source of diversity for blind source separation (BSS), hence extending the MCA to multichannel data. Building on these ingredients, we will provide an overview the generalized MCA introduced by the authors in and as a fast and efficient BSS method. We will illustrate the application of these algorithms on several real examples. We conclude our tour by briefly describing our software toolboxes made available for download on the Internet for sparse signal and image decomposition and separation

Flexible Multi-layer Sparse Approximations of Matrices and Applications

The computational cost of many signal processing and machine learning techniques is often dominated by the cost of applying certain linear operators to high-dimensional vectors. This paper introduces an algorithm aimed at reducing the complexity of applying linear operators in high dimension by approximately factorizing the corresponding matrix into few sparse factors. The approach relies on recent advances in non-convex optimization. It is first explained and analyzed in details and then demonstrated experimentally on various problems including dictionary learning for image denoising, and the approximation of large matrices arising in inverse problems

CT Image Reconstruction by Spatial-Radon Domain Data-Driven Tight Frame Regularization

This paper proposes a spatial-Radon domain CT image reconstruction model based on data-driven tight frames (SRD-DDTF). The proposed SRD-DDTF model combines the idea of joint image and Radon domain inpainting model of \cite{Dong2013X} and that of the data-driven tight frames for image denoising \cite{cai2014data}. It is different from existing models in that both CT image and its corresponding high quality projection image are reconstructed simultaneously using sparsity priors by tight frames that are adaptively learned from the data to provide optimal sparse approximations. An alternative minimization algorithm is designed to solve the proposed model which is nonsmooth and nonconvex. Convergence analysis of the algorithm is provided. Numerical experiments showed that the SRD-DDTF model is superior to the model by \cite{Dong2013X} especially in recovering some subtle structures in the images