Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.Comment: 31 pages, 8 figure

Stability and convergence analysis of transform-domain LMS adaptive filters with second-order autoregressive process

In this paper, the stability and convergence properties of the class of transform-domain least mean square (LMS) adaptive filters with second-order autoregressive (AR) process are investigated. It is well known that this class of adaptive filters improve convergence property of the standard LMS adaptive filters by applying the fixed data-independent orthogonal transforms and power normalization. However, the convergence performance of this class of adaptive filters can be quite different for various input processes, and it has not been fully explored. In this paper, we first discuss the mean-square stability and steady-state performance of this class of adaptive filters. We then analyze the effects of the transforms and power normalization performed in the various adaptive filters for both first-order and second-order AR processes. We derive the input asymptotic eigenvalue distributions and make comparisons on their convergence performance. Finally, computer simulations on AR process as well as moving-average (MA) process and autoregressive-moving-average (ARMA) process are demonstrated for the support of the analytical results.<br /

Stability and convergence analysis of transform-domain LMS adaptive filters with second-order autoregressive process

In this paper, the stability and convergence properties of the class of transform-domain least mean square (LMS) adaptive filters with second-order autoregressive (AR) process are investigated. It is well known that this class of adaptive filters improve convergence property of the standard LMS adaptive filters by applying the fixed data-independent orthogonal transforms and power normalization. However, the convergence performance of this class of adaptive filters can be quite different for various input processes, and it has not been fully explored. In this paper, we first discuss the mean-square stability and steady-state performance of this class of adaptive filters. We then analyze the effects of the transforms and power normalization performed in the various adaptive filters for both first-order and second-order AR processes. We derive the input asymptotic eigenvalue distributions and make comparisons on their convergence performance. Finally, computer simulations on AR process as well as moving-average (MA) process and autoregressive-moving-average (ARMA) process are demonstrated for the support of the analytical results

Contribuições à modelagem estocástica de algoritmos adaptativos normalizados

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Elétrica, Florianópolis, 2015.Este trabalho de pesquisa trata da modelagem estocástica de três algoritmos adaptativos bem conhecidos da literatura, a saber: o algoritmo NLMS (normalized least-mean-square), o algoritmo IAF PNLMS (individual-activation-factor proportionate NLMS) e o algoritmo TDLMS (transform-domain least-mean-square). Particularmente para o algoritmo NLMS, um modelo estocástico analítico é obtido levando em conta um ambiente não estacionário e sinais de entrada gaussianos complexos. Baseado nas expressões de modelo, o impacto dos parâmetros do algoritmo sobre o seu desempenho é discutido, evidenciando algumas das características de rastreamento do algoritmo NLMS frente ao ambiente não estacionário considerado. Para o algoritmo IAF-PNLMS, assumindo um ambiente estacionário, um modelo estocástico mais preciso do que os até então disponíveis na literatura é apresentado, considerando sinais de entrada gaussianos correlacionados tanto complexos quanto reais. Com respeito ao algoritmo TDLMS, um modelo estocástico melhorado é derivado focando em um ambiente não estacionário e sinais de entrada gaussianos correlacionados reais. A partir das expressões de modelo obtidas, o impacto dos parâmetros do algoritmo TDLMS sobre o seu desempenho é discutido. Resultados de simulação para diferentes cenários de operação são mostrados, confirmando a precisão dos modelos estocásticos propostos tanto na fase transitória quanto em regime permanente.Abstract : This research work focuses on the stochastic modeling of three well-known adaptive algorithms from the literature, namely: the normalized least-mean-square (NLMS) algorithm, the individual-activation-factor proportionate NLMS (IAF-PNLMS) algorithm, and the transform-domain least-mean-square (TDLMS) algorithm. Particularly for the NLMS algorithm, an analytical stochastic model is obtained taking into account a nonstationary environment and complex-valued Gaussian input data. Based on the obtained model expressions, the impact of the algorithm parameters on its performance is discussed, clarifying some of the tracking properties of the NLMS algorithm vis-à-vis the nonstationary environment considered. For the IAF-PNLMS algorithm, assuming a stationary environment, a more accurate stochastic model than those available so far in the literature is presented considering both complex- and real-valued Gaussian correlated input data. Regarding the TDLMS algorithm, an improved stochastic model is derived focusing on a nonstationary environment and real-valued Gaussian correlated input data. From the obtained model expressions, the impact of the TDLMS algorithm parameters on its performance is discussed. Simulation results for different operating scenarios are shown, confirming the accuracy of the proposed stochastic models for both transient and steady-state phases