Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.Comment: 31 pages, 8 figure

On the Performance Analysis of a Class of Transform-domain NLMS Algorithms with Gaussian Inputs and Mixture Gaussian Additive Noise Environment

This paper studies the convergence performance of the transform domain normalized least mean square (TDNLMS) algorithm with general nonlinearity and the transform domain normalized least mean M-estimate (TDNLMM) algorithm in Gaussian inputs and additive Gaussian and impulsive noise environment. The TDNLMM algorithm, which is derived from robust M-estimation, has the advantage of improved performance over the conventional TDNLMS algorithm in combating impulsive noises. Using Price's theorem and its extension, the above algorithms can be treated in a single framework respectively for Gaussian and impulsive noise environments. Further, by introducing new special integral functions, related expectations can be evaluated so as to obtain decoupled difference equations which describe the mean and mean square behaviors of the TDNLMS and TDNLMM algorithms. These analytical results reveal the advantages of the TDNLMM algorithm in impulsive noise environment, and are in good agreement with computer simulation results. © 2010 The Author(s).published_or_final_versionSpringer Open Choice, 21 Feb 201

Improved approximate QR-LS algorithms for adaptive filtering

This paper studies a class of O(N) approximate QR-based least squares (A-QR-LS) algorithm recently proposed by Liu in 1995. It is shown that the A-QR-LS algorithm is equivalent to a normalized LMS algorithm with time-varying stepsizes and element-wise normalization of the input signal vector. It reduces to the QR-LMS algorithm proposed by Liu et al. in 1998, when all the normalization constants are chosen as the Euclidean norm of the input signal vector. An improved transform-domain approximate QR-LS (TA-QR-LS) algorithm, where the input signal vector is first approximately decorrelated by some unitary transformations before the normalization, is proposed to improve its convergence for highly correlated signals. The mean weight vectors of the algorithms are shown to converge to the optimal Wiener solution if the weighting factor w of the algorithm is chosen between 0 and 1. New Givens rotations-based algorithms for the A-QR-LS, TA-QR-LS, and the QR-LMS algorithms are proposed to reduce their arithmetic complexities. This reduces the arithmetic complexity by a factor of 2, and allows square root-free versions of the algorithms be developed. The performances of the various algorithms are evaluated through computer simulation of a system identification problem and an acoustic echo canceller. © 2004 IEEE.published_or_final_versio

A new switch-mode noise-constrained transform domain NLMS adaptive filtering algorithm

The transform domain normalized least mean squares (TDNLMS) algorithm is an efficient adaptive algorithm, which offers fast convergence speed with a reasonably low arithmetic complexity. However, its convergence speed is usually limited by the fixed step-size so as to achieve a low desired misadjustment. In this paper a new switch-mode noise-constrained TDNLMS (SNC-TDNLMS) algorithm is proposed. It employs a maximum step-size mode in initial convergence and a noise-constrained mode afterwards to improve the convergence speed and steady-state performance. The mean and mean square convergence behaviors of the proposed algorithm are studied to characterize its convergence condition and steady-state excess mean square error (EMSE). Based on the theoretical results, an automatic threshold selection scheme for mode switching is developed. Computer simulations are conducted to show the effectiveness of the proposed algorithm and verify the theoretical results. © 2011 IEEE.published_or_final_versionThe 2011 IEEE International Symposium on Circuits and Systems (ISCAS), Rio de Janeiro, Brazil, 15-18 May 2011. In Proceedings of ISCAS, 2011, p. 117-12

A New Variable Regularized Transform Domain NLMS Adaptive Filtering Algorithm-Acoustic Applications and Performance Analysis

published_or_final_versio

A new regularized transform-domain NLMS adaptive filtering algorithm

The transform domain normalized LMS (TD-NLMS)-adaptive filtering algorithm is an efficient adaptive filter with fast convergence speed and reasonably low arithmetic complexity. However, it is sensitive to the level of the excitation signal, which may vary significantly over time in speech and audio signals. This paper proposes a new regularized transform domain NLMS (R-TDNLMS) algorithm and studies its mean and mean square convergence performance. The proposed algorithm extends the conventional TDNLMS algorithm by imposing a regularization term on the coefficients to reduce the variance of the estimator. The mean and mean square convergence behaviors of the proposed algorithm are studied to characterize its convergence condition and steady-state excess mean squares error (MSE). It shows that regularization can help to reduce the MSE for coloured inputs by trading slight bias for variance. Moreover, the immunity to varying input signal level is significantly reduced. Computer simulations are conducted to examine the effectiveness of the proposed algorithm and they are in good agreement with the theoretical analysis. © 2010 IEEE.published_or_final_versionThe 2010 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS 2010), Kuala Lumpur, Malaysia, 6-9 December 2010. In Proceedings of APCCAS, 2010, p. 696-69

New sequential partial-update least mean M-estimate algorithms for robust adaptive system identification in impulsive noise

The sequential partial-update least mean square (S-LMS)-based algorithms are efficient methods for reducing the arithmetic complexity in adaptive system identification and other industrial informatics applications. They are also attractive in acoustic applications where long impulse responses are encountered. A limitation of these algorithms is their degraded performances in an impulsive noise environment. This paper proposes new robust counterparts for the S-LMS family based on M-estimation. The proposed sequential least mean M-estimate (S-LMM) family of algorithms employ nonlinearity to improve their robustness to impulsive noise. Another contribution of this paper is the presentation of a convergence performance analysis for the S-LMS/S-LMM family for Gaussian inputs and additive Gaussian or contaminated Gaussian noises. The analysis is important for engineers to understand the behaviors of these algorithms and to select appropriate parameters for practical realizations. The theoretical analyses reveal the advantages of input normalization and the M-estimation in combating impulsive noise. Computer simulations on system identification and joint active noise and acoustic echo cancellations in automobiles with double-talk are conducted to verify the theoretical results and the effectiveness of the proposed algorithms. © 2010 IEEE.published_or_final_versio