Stochastic Behavior of the Nonnegative Least Mean Fourth Algorithm for Stationary Gaussian Inputs and Slow Learning

Some system identification problems impose nonnegativity constraints on the parameters to estimate due to inherent physical characteristics of the unknown system. The nonnegative least-mean-square (NNLMS) algorithm and its variants allow to address this problem in an online manner. A nonnegative least mean fourth (NNLMF) algorithm has been recently proposed to improve the performance of these algorithms in cases where the measurement noise is not Gaussian. This paper provides a first theoretical analysis of the stochastic behavior of the NNLMF algorithm for stationary Gaussian inputs and slow learning. Simulation results illustrate the accuracy of the proposed analysis.Comment: 11 pages, 8 figures, submitted for publicatio

A Novel Family of Adaptive Filtering Algorithms Based on The Logarithmic Cost

We introduce a novel family of adaptive filtering algorithms based on a relative logarithmic cost. The new family intrinsically combines the higher and lower order measures of the error into a single continuous update based on the error amount. We introduce important members of this family of algorithms such as the least mean logarithmic square (LMLS) and least logarithmic absolute difference (LLAD) algorithms that improve the convergence performance of the conventional algorithms. However, our approach and analysis are generic such that they cover other well-known cost functions as described in the paper. The LMLS algorithm achieves comparable convergence performance with the least mean fourth (LMF) algorithm and extends the stability bound on the step size. The LLAD and least mean square (LMS) algorithms demonstrate similar convergence performance in impulse-free noise environments while the LLAD algorithm is robust against impulsive interferences and outperforms the sign algorithm (SA). We analyze the transient, steady state and tracking performance of the introduced algorithms and demonstrate the match of the theoretical analyzes and simulation results. We show the extended stability bound of the LMLS algorithm and analyze the robustness of the LLAD algorithm against impulsive interferences. Finally, we demonstrate the performance of our algorithms in different scenarios through numerical examples.Comment: Submitted to IEEE Transactions on Signal Processin

Analysis and Evaluation of the Family of Sign Adaptive Algorithms

In this thesis, four novel sign adaptive algorithms proposed by the author were analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Fourth (SRLMF), Sign Regressor Least Mean Mixed-Norm (SRLMMN), Normalized Sign Regressor Least Mean Fourth (NSRLMF), and Normalized Sign Regressor Least Mean Mixed-Norm (NSRLMMN). The performance of the latter three algorithms has been analyzed and evaluated for real-valued data only. While the performance of the SRLMF algorithm has been analyzed and evaluated for both cases of real- and complex-valued data. Additionally, four sign adaptive algorithms proposed by other researchers were also analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Square (SRLMS), Sign-Sign Least Mean Square (SSLMS), Normalized Sign-Error Least Mean Square (NSLMS), and Normalized Sign Regressor Least Mean Square (NSRLMS). The performance of the latter three algorithms has been analyzed and evaluated for both cases of real- and complex-valued data. While the performance of the SRLMS algorithm has been analyzed and evaluated for complex-valued data only. The framework employed in this thesis relies on energy conservation approach. The energy conservation framework has been applied uniformly for the evaluation of the performance of the aforementioned eight sign adaptive algorithms proposed by the author and other researchers. In other words, the energy conservation framework stands out as a common theme that runs throughout the treatment of the performance of the aforementioned eight algorithms. Some of the results from the performance evaluation of the four novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, NSRLMF, and NSRLMMN are as follows. It was shown that the convergence performance of the SRLMF and SRLMMN algorithms for real-valued data was similar to those of the Least Mean Fourth (LMF) and Least Mean Mixed-Norm (LMMN) algorithms, respectively. Moreover, it was also shown that the NSRLMF and NSRLMMN algorithms exhibit a compromised convergence performance for realvalued data as compared to the Normalized Least Mean Fourth (NLMF) and Normalized Least Mean Mixed-Norm (NLMMN) algorithms, respectively. Some misconceptions among biomedical signal processing researchers concerning the implementation of adaptive noise cancelers using the Sign-Error Least Mean Fourth (SLMF), Sign-Sign Least Mean Fourth (SSLMF), and their variant algorithms were also removed. Finally, three of the novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, and NSRLMF have been successfully employed by other researchers and the author in applications ranging from power quality improvement in the distribution system and multiple artifacts removal from various physiological signals such as ElectroCardioGram (ECG) and ElectroEncephaloGram (EEG)

Algorithms and structures for long adaptive echo cancellers

The main theme of this thesis is adaptive echo cancellation. Two novel independent approaches are proposed for the design of long echo cancellers with improved performance. In the first approach, we present a novel structure for bulk delay estimation in long echo cancellers which considerably reduces the amount of excess error. The miscalculation of the delay between the near-end and the far-end sections is one of the main causes of this excess error. Two analyses, based on the Least Mean Squares (LMS) algorithm, are presented where certain shapes for the transitions between the end of the near-end section and the beginning of the far-end one are considered. Transient and steady-state behaviours and convergence conditions for the proposed algorithm are studied. Comparisons between the algorithms developed for each transition are presented, and the simulation results agree well with the theoretical derivations. In the second approach, a generalised performance index is proposed for the design of the echo canceller. The proposed algorithm consists of simultaneously applying the LMS algorithm to the near-end section and the Least Mean Fourth (LMF) algorithm to the far-end section of the echo canceller. This combination results in a substantial improvement of the performance of the proposed scheme over both the LMS and other algorithms proposed for comparison. In this approach, the proposed algorithm will be henceforth called the Least Mean Mixed-Norm (LMMN) algorithm. The advantages of the LMMN algorithm over previously reported ones are two folds: it leads to a faster convergence and results in a smaller misadjustment error. Finally, the convergence properties of the LMMN algorithm are derived and the simulation results confirm the superior performance of this proposed algorithm over other well known algorithms

Analysis of the sign regressor least mean fourth adaptive algorithm

A novel algorithm, called the signed regressor least mean fourth (SRLMF) adaptive algorithm, that reduces the computational cost and complexity while maintaining good performance is presented. Expressions are derived for the steady-state excess-mean-square error (EMSE) of the SRLMF algorithm in a stationary environment. A sufficient condition for the convergence in the mean of the SRLMF algorithm is derived. Also, expressions are obtained for the tracking EMSE of the SRLMF algorithm in a nonstationary environment, and consequently an optimum value of the step-size is obtained. Moreover, the weighted variance relation has been extended in order to derive expressions for the mean-square error (MSE) and the mean-square deviation (MSD) of the proposed algorithm during the transient phase. Computer simulations are carried out to corroborate the theoretical findings. It is shown that there is a good match between the theoretical and simulated results. It is also shown that the SRLMF algorithm has no performance degradation when compared with the least mean fourth (LMF) algorithm. The results in this study emphasize the usefulness of this algorithm in applications requiring reduced implementation costs for which the LMF algorithm is too complex