Residual Ratio Thresholding for Model Order Selection

Model order selection (MOS) in linear regression models is a widely studied problem in signal processing. Techniques based on information theoretic criteria (ITC) are algorithms of choice in MOS problems. This article proposes a novel technique called residual ratio thresholding for MOS in linear regression models which is fundamentally different from the ITC based MOS criteria widely discussed in literature. This article also provides a rigorous mathematical analysis of the high signal to noise ratio (SNR) and large sample size behaviour of RRT. RRT is numerically shown to deliver a highly competitive performance when compared to popular model order selection criteria like Akaike information criterion (AIC), Bayesian information criterion (BIC), penalised adaptive likelihood (PAL) etc. especially when the sample size is small.Comment: 13 pages, 23 figure

High SNR Consistent Compressive Sensing Without Signal and Noise Statistics

Recovering the support of sparse vectors in underdetermined linear regression models, \textit{aka}, compressive sensing is important in many signal processing applications. High SNR consistency (HSC), i.e., the ability of a support recovery technique to correctly identify the support with increasing signal to noise ratio (SNR) is an increasingly popular criterion to qualify the high SNR optimality of support recovery techniques. The HSC results available in literature for support recovery techniques applicable to underdetermined linear regression models like least absolute shrinkage and selection operator (LASSO), orthogonal matching pursuit (OMP) etc. assume \textit{a priori} knowledge of noise variance or signal sparsity. However, both these parameters are unavailable in most practical applications. Further, it is extremely difficult to estimate noise variance or signal sparsity in underdetermined regression models. This limits the utility of existing HSC results. In this article, we propose two techniques, \textit{viz.}, residual ratio minimization (RRM) and residual ratio thresholding with adaptation (RRTA) to operate OMP algorithm without the \textit{a priroi} knowledge of noise variance and signal sparsity and establish their HSC analytically and numerically. To the best of our knowledge, these are the first and only noise statistics oblivious algorithms to report HSC in underdetermined regression models.Comment: 13 pages, 6 figure