Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.Comment: 31 pages, 8 figure

Analysis and Evaluation of the Family of Sign Adaptive Algorithms

In this thesis, four novel sign adaptive algorithms proposed by the author were analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Fourth (SRLMF), Sign Regressor Least Mean Mixed-Norm (SRLMMN), Normalized Sign Regressor Least Mean Fourth (NSRLMF), and Normalized Sign Regressor Least Mean Mixed-Norm (NSRLMMN). The performance of the latter three algorithms has been analyzed and evaluated for real-valued data only. While the performance of the SRLMF algorithm has been analyzed and evaluated for both cases of real- and complex-valued data. Additionally, four sign adaptive algorithms proposed by other researchers were also analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Square (SRLMS), Sign-Sign Least Mean Square (SSLMS), Normalized Sign-Error Least Mean Square (NSLMS), and Normalized Sign Regressor Least Mean Square (NSRLMS). The performance of the latter three algorithms has been analyzed and evaluated for both cases of real- and complex-valued data. While the performance of the SRLMS algorithm has been analyzed and evaluated for complex-valued data only. The framework employed in this thesis relies on energy conservation approach. The energy conservation framework has been applied uniformly for the evaluation of the performance of the aforementioned eight sign adaptive algorithms proposed by the author and other researchers. In other words, the energy conservation framework stands out as a common theme that runs throughout the treatment of the performance of the aforementioned eight algorithms. Some of the results from the performance evaluation of the four novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, NSRLMF, and NSRLMMN are as follows. It was shown that the convergence performance of the SRLMF and SRLMMN algorithms for real-valued data was similar to those of the Least Mean Fourth (LMF) and Least Mean Mixed-Norm (LMMN) algorithms, respectively. Moreover, it was also shown that the NSRLMF and NSRLMMN algorithms exhibit a compromised convergence performance for realvalued data as compared to the Normalized Least Mean Fourth (NLMF) and Normalized Least Mean Mixed-Norm (NLMMN) algorithms, respectively. Some misconceptions among biomedical signal processing researchers concerning the implementation of adaptive noise cancelers using the Sign-Error Least Mean Fourth (SLMF), Sign-Sign Least Mean Fourth (SSLMF), and their variant algorithms were also removed. Finally, three of the novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, and NSRLMF have been successfully employed by other researchers and the author in applications ranging from power quality improvement in the distribution system and multiple artifacts removal from various physiological signals such as ElectroCardioGram (ECG) and ElectroEncephaloGram (EEG)