The Krylov-proportionate normalized least mean fourth approach: Formulation and performance analysis

Cataloged from PDF version of article.We propose novel adaptive filtering algorithms based on the mean-fourth error objective while providing further improvements on the convergence performance through proportionate update. We exploit the sparsity of the system in the mean-fourth error framework through the proportionate normalized least mean fourth (PNLMF) algorithm. In order to broaden the applicability of the PNLMF algorithm to dispersive (non-sparse) systems, we introduce the Krylov-proportionate normalized least mean fourth (KPNLMF) algorithm using the Krylov subspace projection technique. We propose the Krylov-proportionate normalized least mean mixed norm (KPNLMMN) algorithm combining the mean-square and mean-fourth error objectives in order to enhance the performance of the constituent filters. Additionally, we propose the stable-PNLMF and stable-KPNLMF algorithms overcoming the stability issues induced due to the usage of the mean fourth error framework. Finally, we provide a complete performance analysis, i.e., the transient and the steady-state analyses, for the proportionate update based algorithms, e.g., the PNLMF, the KPNLMF algorithms and their variants; and analyze their tracking performance in a non-stationary environment. Through the numerical examples, we demonstrate the match of the theoretical and ensemble averaged results and show the superior performance of the introduced algorithms in different scenarios. (C) 2014 Elsevier B.V. All rights reserved

A Novel Family of Adaptive Filtering Algorithms Based on The Logarithmic Cost

We introduce a novel family of adaptive filtering algorithms based on a relative logarithmic cost. The new family intrinsically combines the higher and lower order measures of the error into a single continuous update based on the error amount. We introduce important members of this family of algorithms such as the least mean logarithmic square (LMLS) and least logarithmic absolute difference (LLAD) algorithms that improve the convergence performance of the conventional algorithms. However, our approach and analysis are generic such that they cover other well-known cost functions as described in the paper. The LMLS algorithm achieves comparable convergence performance with the least mean fourth (LMF) algorithm and extends the stability bound on the step size. The LLAD and least mean square (LMS) algorithms demonstrate similar convergence performance in impulse-free noise environments while the LLAD algorithm is robust against impulsive interferences and outperforms the sign algorithm (SA). We analyze the transient, steady state and tracking performance of the introduced algorithms and demonstrate the match of the theoretical analyzes and simulation results. We show the extended stability bound of the LMLS algorithm and analyze the robustness of the LLAD algorithm against impulsive interferences. Finally, we demonstrate the performance of our algorithms in different scenarios through numerical examples.Comment: Submitted to IEEE Transactions on Signal Processin

Time-varying signal processing using multi-wavelet basis functions and a modified block least mean square algorithm

This paper introduces a novel parametric modeling and identification method for linear time-varying systems using a modified block least mean square (LMS) approach where the time-varying parameters are approximated using multi-wavelet basis functions. This approach can be used to track rapidly or even sharply varying processes and is more suitable for recursive estimation of process parameters by combining wavelet approximation theory with a modified block LMS algorithm. Numerical examples are provided to show the effectiveness of the proposed method for dealing with severely nonstatinoary processes

Analysis and Evaluation of the Family of Sign Adaptive Algorithms

In this thesis, four novel sign adaptive algorithms proposed by the author were analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Fourth (SRLMF), Sign Regressor Least Mean Mixed-Norm (SRLMMN), Normalized Sign Regressor Least Mean Fourth (NSRLMF), and Normalized Sign Regressor Least Mean Mixed-Norm (NSRLMMN). The performance of the latter three algorithms has been analyzed and evaluated for real-valued data only. While the performance of the SRLMF algorithm has been analyzed and evaluated for both cases of real- and complex-valued data. Additionally, four sign adaptive algorithms proposed by other researchers were also analyzed and evaluated for floating-point arithmetic operations. These four algorithms include Sign Regressor Least Mean Square (SRLMS), Sign-Sign Least Mean Square (SSLMS), Normalized Sign-Error Least Mean Square (NSLMS), and Normalized Sign Regressor Least Mean Square (NSRLMS). The performance of the latter three algorithms has been analyzed and evaluated for both cases of real- and complex-valued data. While the performance of the SRLMS algorithm has been analyzed and evaluated for complex-valued data only. The framework employed in this thesis relies on energy conservation approach. The energy conservation framework has been applied uniformly for the evaluation of the performance of the aforementioned eight sign adaptive algorithms proposed by the author and other researchers. In other words, the energy conservation framework stands out as a common theme that runs throughout the treatment of the performance of the aforementioned eight algorithms. Some of the results from the performance evaluation of the four novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, NSRLMF, and NSRLMMN are as follows. It was shown that the convergence performance of the SRLMF and SRLMMN algorithms for real-valued data was similar to those of the Least Mean Fourth (LMF) and Least Mean Mixed-Norm (LMMN) algorithms, respectively. Moreover, it was also shown that the NSRLMF and NSRLMMN algorithms exhibit a compromised convergence performance for realvalued data as compared to the Normalized Least Mean Fourth (NLMF) and Normalized Least Mean Mixed-Norm (NLMMN) algorithms, respectively. Some misconceptions among biomedical signal processing researchers concerning the implementation of adaptive noise cancelers using the Sign-Error Least Mean Fourth (SLMF), Sign-Sign Least Mean Fourth (SSLMF), and their variant algorithms were also removed. Finally, three of the novel sign adaptive algorithms proposed by the author, namely SRLMF, SRLMMN, and NSRLMF have been successfully employed by other researchers and the author in applications ranging from power quality improvement in the distribution system and multiple artifacts removal from various physiological signals such as ElectroCardioGram (ECG) and ElectroEncephaloGram (EEG)

A Comprehensive Performance Evaluation of Deformable Face Tracking "In-the-Wild"

Recently, technologies such as face detection, facial landmark localisation and face recognition and verification have matured enough to provide effective and efficient solutions for imagery captured under arbitrary conditions (referred to as "in-the-wild"). This is partially attributed to the fact that comprehensive "in-the-wild" benchmarks have been developed for face detection, landmark localisation and recognition/verification. A very important technology that has not been thoroughly evaluated yet is deformable face tracking "in-the-wild". Until now, the performance has mainly been assessed qualitatively by visually assessing the result of a deformable face tracking technology on short videos. In this paper, we perform the first, to the best of our knowledge, thorough evaluation of state-of-the-art deformable face tracking pipelines using the recently introduced 300VW benchmark. We evaluate many different architectures focusing mainly on the task of on-line deformable face tracking. In particular, we compare the following general strategies: (a) generic face detection plus generic facial landmark localisation, (b) generic model free tracking plus generic facial landmark localisation, as well as (c) hybrid approaches using state-of-the-art face detection, model free tracking and facial landmark localisation technologies. Our evaluation reveals future avenues for further research on the topic.Comment: E. Antonakos and P. Snape contributed equally and have joint second authorshi

A Robust Zero-point Attraction LMS Algorithm on Near Sparse System Identification

The newly proposed $l_1$ norm constraint zero-point attraction Least Mean Square algorithm (ZA-LMS) demonstrates excellent performance on exact sparse system identification. However, ZA-LMS has less advantage against standard LMS when the system is near sparse. Thus, in this paper, firstly the near sparse system modeling by Generalized Gaussian Distribution is recommended, where the sparsity is defined accordingly. Secondly, two modifications to the ZA-LMS algorithm have been made. The $l_1$ norm penalty is replaced by a partial $l_1$ norm in the cost function, enhancing robustness without increasing the computational complexity. Moreover, the zero-point attraction item is weighted by the magnitude of estimation error which adjusts the zero-point attraction force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS (DWZA-LMS) algorithm is further proposed, which shows better performance on near sparse system identification. In addition, the mean square performance of DWZA-LMS algorithm is analyzed. Finally, computer simulations demonstrate the effectiveness of the proposed algorithm and verify the result of theoretical analysis.Comment: 20 pages, 11 figure