A Robust Zero-point Attraction LMS Algorithm on Near Sparse System Identification

The newly proposed $l_1$ norm constraint zero-point attraction Least Mean Square algorithm (ZA-LMS) demonstrates excellent performance on exact sparse system identification. However, ZA-LMS has less advantage against standard LMS when the system is near sparse. Thus, in this paper, firstly the near sparse system modeling by Generalized Gaussian Distribution is recommended, where the sparsity is defined accordingly. Secondly, two modifications to the ZA-LMS algorithm have been made. The $l_1$ norm penalty is replaced by a partial $l_1$ norm in the cost function, enhancing robustness without increasing the computational complexity. Moreover, the zero-point attraction item is weighted by the magnitude of estimation error which adjusts the zero-point attraction force dynamically. By combining the two improvements, Dynamic Windowing ZA-LMS (DWZA-LMS) algorithm is further proposed, which shows better performance on near sparse system identification. In addition, the mean square performance of DWZA-LMS algorithm is analyzed. Finally, computer simulations demonstrate the effectiveness of the proposed algorithm and verify the result of theoretical analysis.Comment: 20 pages, 11 figure

Stereophonic acoustic echo cancellation employing selective-tap adaptive algorithms

Published versio

Performance Analysis of l_0 Norm Constraint Least Mean Square Algorithm

As one of the recently proposed algorithms for sparse system identification, $l_0$ norm constraint Least Mean Square ($l_0$-LMS) algorithm modifies the cost function of the traditional method with a penalty of tap-weight sparsity. The performance of $l_0$-LMS is quite attractive compared with its various precursors. However, there has been no detailed study of its performance. This paper presents all-around and throughout theoretical performance analysis of $l_0$-LMS for white Gaussian input data based on some reasonable assumptions. Expressions for steady-state mean square deviation (MSD) are derived and discussed with respect to algorithm parameters and system sparsity. The parameter selection rule is established for achieving the best performance. Approximated with Taylor series, the instantaneous behavior is also derived. In addition, the relationship between $l_0$-LMS and some previous arts and the sufficient conditions for $l_0$-LMS to accelerate convergence are set up. Finally, all of the theoretical results are compared with simulations and are shown to agree well in a large range of parameter setting.Comment: 31 pages, 8 figure

System Identification with Applications in Speech Enhancement

As the increasing popularity of integrating hands-free telephony on mobile portable devices and the rapid development of voice over internet protocol, identification of acoustic systems has become desirable for compensating distortions introduced to speech signals during transmission, and hence enhancing the speech quality. The objective of this research is to develop system identification algorithms for speech enhancement applications including network echo cancellation and speech dereverberation. A supervised adaptive algorithm for sparse system identification is developed for network echo cancellation. Based on the framework of selective-tap updating scheme on the normalized least mean squares algorithm, the MMax and sparse partial update tap-selection strategies are exploited in the frequency domain to achieve fast convergence performance with low computational complexity. Through demonstrating how the sparseness of the network impulse response varies in the transformed domain, the multidelay filtering structure is incorporated to reduce the algorithmic delay. Blind identification of SIMO acoustic systems for speech dereverberation in the presence of common zeros is then investigated. First, the problem of common zeros is defined and extended to include the presence of near-common zeros. Two clustering algorithms are developed to quantify the number of these zeros so as to facilitate the study of their effect on blind system identification and speech dereverberation. To mitigate such effect, two algorithms are developed where the two-stage algorithm based on channel decomposition identifies common and non-common zeros sequentially; and the forced spectral diversity approach combines spectral shaping filters and channel undermodelling for deriving a modified system that leads to an improved dereverberation performance. Additionally, a solution to the scale factor ambiguity problem in subband-based blind system identification is developed, which motivates further research on subbandbased dereverberation techniques. Comprehensive simulations and discussions demonstrate the effectiveness of the aforementioned algorithms. A discussion on possible directions of prospective research on system identification techniques concludes this thesis

On data-selective learning

Adaptive filters are applied in several electronic and communication devices like smartphones, advanced headphones, DSP chips, smart antenna, and teleconference systems. Also, they have application in many areas such as system identification, channel equalization, noise reduction, echo cancellation, interference cancellation, signal prediction, and stock market. Therefore, reducing the energy consumption of the adaptive filtering algorithms has great importance, particularly in green technologies and in devices using battery. In this thesis, data-selective adaptive filters, in particular the set-membership (SM) adaptive filters, are the tools to reach the goal. There are well known SM adaptive filters in literature. This work introduces new algorithms based on the classical ones in order to improve their performances and reduce the number of required arithmetic operations at the same time. Therefore, firstly, we analyze the robustness of the classical SM adaptive filtering algorithms. Secondly, we extend the SM technique to trinion and quaternion systems. Thirdly, by combining SM filtering and partialupdating, we introduce a new improved set-membership affine projection algorithm with constrained step size to improve its stability behavior. Fourthly, we propose some new least-mean-square (LMS) based and recursive least-squares based adaptive filtering algorithms with low computational complexity for sparse systems. Finally, we derive some feature LMS algorithms to exploit the hidden sparsity in the parameters.Filtros adaptativos são aplicados em diversos aparelhos eletrônicos e de comunicação, como smartphones, fone de ouvido avançados, DSP chips, antenas inteligentes e sistemas de teleconferência. Eles também têm aplicação em várias áreas como identificação de sistemas, equalização de canal, cancelamento de eco, cancelamento de interferência, previsão de sinal e mercado de ações. Desse modo, reduzir o consumo de energia de algoritmos adaptativos tem importância significativa, especialmente em tecnologias verdes e aparelhos que usam bateria. Nesta tese, filtros adaptativos com seleção de dados, em particular filtros adaptativos da família set-membership (SM), são apresentados para cumprir essa missão. No presente trabalho objetivamos apresentar novos algoritmos, baseados nos clássicos, a fim de aperfeiçoar seus desempenhos e, ao mesmo tempo, reduzir o número de operações aritméticas exigidas. Dessa forma, primeiro analisamos a robustez dos filtros adaptativos SM clássicos. Segundo, estendemos o SM aos números trinions e quaternions. Terceiro, foram utilizadas também duas famílias de algoritmos, SM filtering e partial-updating, de uma maneira elegante, visando reduzir energia ao máximo possível e obter um desempenho competitivo em termos de estabilidade. Quarto, a tese propõe novos filtros adaptativos baseado em algoritmos least-mean-square (LMS) e mínimos quadrados recursivos com complexidade computacional baixa para espaços esparsos. Finalmente, derivamos alguns algoritmos feature LMS para explorar a esparsidade escondida nos parâmetros

Collaborative adaptive filtering for machine learning

Quantitative performance criteria for the analysis of machine learning architectures and algorithms have long been established. However, qualitative performance criteria, which identify fundamental signal properties and ensure any processing preserves the desired properties, are still emerging. In many cases, whilst offline statistical tests exist such as assessment of nonlinearity or stochasticity, online tests which not only characterise but also track changes in the nature of the signal are lacking. To that end, by employing recent developments in signal characterisation, criteria are derived for the assessment of the changes in the nature of the processed signal. Through the fusion of the outputs of adaptive filters a single collaborative hybrid filter is produced. By tracking the dynamics of the mixing parameter of this filter, rather than the actual filter performance, a clear indication as to the current nature of the signal is given. Implementations of the proposed method show that it is possible to quantify the degree of nonlinearity within both real- and complex-valued data. This is then extended (in the real domain) from dealing with nonlinearity in general, to a more specific example, namely sparsity. Extensions of adaptive filters from the real to the complex domain are non-trivial and the differences between the statistics in the real and complex domains need to be taken into account. In terms of signal characteristics, nonlinearity can be both split- and fully-complex and complex-valued data can be considered circular or noncircular. Furthermore, by combining the information obtained from hybrid filters of different natures it is possible to use this method to gain a more complete understanding of the nature of the nonlinearity within a signal. This also paves the way for building multidimensional feature spaces and their application in data/information fusion. To produce online tests for sparsity, adaptive filters for sparse environments are investigated and a unifying framework for the derivation of proportionate normalised least mean square (PNLMS) algorithms is presented. This is then extended to derive variants with an adaptive step-size. In order to create an online test for noncircularity, a study of widely linear autoregressive modelling is presented, from which a proof of the convergence of the test for noncircularity can be given. Applications of this method are illustrated on examples such as biomedical signals, speech and wind data

Zero attracting recursive least squares algorithms

The l1-norm sparsity constraint is a widely used technique for constructing sparse models. In this contribution, two zero-attracting recursive least squares algorithms, referred to as ZA-RLS-I and ZA-RLS-II, are derived by employing the l1-norm of parameter vector constraint to facilitate the model sparsity. In order to achieve a closed-form solution, the l1-norm of the parameter vector is approximated by an adaptively weighted l2-norm, in which the weighting factors are set as the inversion of the associated l1-norm of parameter estimates that are readily available in the adaptive learning environment. ZA-RLS-II is computationally more efficient than ZA-RLS-I by exploiting the known results from linear algebra as well as the sparsity of the system. The proposed algorithms are proven to converge, and adaptive sparse channel estimation is used to demonstrate the effectiveness of the proposed approach