Stochastic analysis of an error power ratio scheme applied to the affine combination of two LMS adaptive filters

The affine combination of two adaptive filters that simultaneously adapt on the same inputs has been actively investigated. In these structures, the filter outputs are linearly combined to yield a performance that is better than that of either filter. Various decision rules can be used to determine the time-varying parameter for combining the filter outputs. A recently proposed scheme based on the ratio of error powers of the two filters has been shown by simulation to achieve nearly optimum performance. The purpose of this paper is to present a first analysis of the statistical behavior of this error power scheme for white Gaussian inputs. Expressions are derived for the mean behavior of the combination parameter and for the adaptive weight mean-square deviation. Monte Carlo simulations show good to excellent agreement with the theoretical predictions

System approach to robust acoustic echo cancellation through semi-blind source separation based on independent component analysis

We live in a dynamic world full of noises and interferences. The conventional acoustic echo cancellation (AEC) framework based on the least mean square (LMS) algorithm by itself lacks the ability to handle many secondary signals that interfere with the adaptive filtering process, e.g., local speech and background noise. In this dissertation, we build a foundation for what we refer to as the system approach to signal enhancement as we focus on the AEC problem. We first propose the residual echo enhancement (REE) technique that utilizes the error recovery nonlinearity (ERN) to "enhances" the filter estimation error prior to the filter adaptation. The single-channel AEC problem can be viewed as a special case of semi-blind source separation (SBSS) where one of the source signals is partially known, i.e., the far-end microphone signal that generates the near-end acoustic echo. SBSS optimized via independent component analysis (ICA) leads to the system combination of the LMS algorithm with the ERN that allows for continuous and stable adaptation even during double talk. Second, we extend the system perspective to the decorrelation problem for AEC, where we show that the REE procedure can be applied effectively in a multi-channel AEC (MCAEC) setting to indirectly assist the recovery of lost AEC performance due to inter-channel correlation, known generally as the "non-uniqueness" problem. We develop a novel, computationally efficient technique of frequency-domain resampling (FDR) that effectively alleviates the non-uniqueness problem directly while introducing minimal distortion to signal quality and statistics. We also apply the system approach to the multi-delay filter (MDF) that suffers from the inter-block correlation problem. Finally, we generalize the MCAEC problem in the SBSS framework and discuss many issues related to the implementation of an SBSS system. We propose a constrained batch-online implementation of SBSS that stabilizes the convergence behavior even in the worst case scenario of a single far-end talker along with the non-uniqueness condition on the far-end mixing system. The proposed techniques are developed from a pragmatic standpoint, motivated by real-world problems in acoustic and audio signal processing. Generalization of the orthogonality principle to the system level of an AEC problem allows us to relate AEC to source separation that seeks to maximize the independence, hence implicitly the orthogonality, not only between the error signal and the far-end signal, but rather, among all signals involved. The system approach, for which the REE paradigm is just one realization, enables the encompassing of many traditional signal enhancement techniques in analytically consistent yet practically effective manner for solving the enhancement problem in a very noisy and disruptive acoustic mixing environment.PhDCommittee Chair: Biing-Hwang Juang; Committee Member: Brani Vidakovic; Committee Member: David V. Anderson; Committee Member: Jeff S. Shamma; Committee Member: Xiaoli M

Collaborative adaptive filtering for machine learning

Quantitative performance criteria for the analysis of machine learning architectures and algorithms have long been established. However, qualitative performance criteria, which identify fundamental signal properties and ensure any processing preserves the desired properties, are still emerging. In many cases, whilst offline statistical tests exist such as assessment of nonlinearity or stochasticity, online tests which not only characterise but also track changes in the nature of the signal are lacking. To that end, by employing recent developments in signal characterisation, criteria are derived for the assessment of the changes in the nature of the processed signal. Through the fusion of the outputs of adaptive filters a single collaborative hybrid filter is produced. By tracking the dynamics of the mixing parameter of this filter, rather than the actual filter performance, a clear indication as to the current nature of the signal is given. Implementations of the proposed method show that it is possible to quantify the degree of nonlinearity within both real- and complex-valued data. This is then extended (in the real domain) from dealing with nonlinearity in general, to a more specific example, namely sparsity. Extensions of adaptive filters from the real to the complex domain are non-trivial and the differences between the statistics in the real and complex domains need to be taken into account. In terms of signal characteristics, nonlinearity can be both split- and fully-complex and complex-valued data can be considered circular or noncircular. Furthermore, by combining the information obtained from hybrid filters of different natures it is possible to use this method to gain a more complete understanding of the nature of the nonlinearity within a signal. This also paves the way for building multidimensional feature spaces and their application in data/information fusion. To produce online tests for sparsity, adaptive filters for sparse environments are investigated and a unifying framework for the derivation of proportionate normalised least mean square (PNLMS) algorithms is presented. This is then extended to derive variants with an adaptive step-size. In order to create an online test for noncircularity, a study of widely linear autoregressive modelling is presented, from which a proof of the convergence of the test for noncircularity can be given. Applications of this method are illustrated on examples such as biomedical signals, speech and wind data

Adaptive polynomial filters

Journal ArticleWhile linear filter are useful in a large number of applications and relatively simple from conceptual and implementational view points. there are many practical situations that require nonlinear processing of the signals involved. This article explains adaptive nonlinear filters equipped with polynomial models of nonlinearity. The polynomial systems considered are those nonlinear systems whose output signals can be related to the input signals through a truncated Volterra series expansion, or a recursive nonlinear difference equation. The Volterra series expansion can model a large class of nonlinear systems and is attractive in filtering applications because the expansion is a linear combination of nonlinear functions of the input signal. The basic ideas behind the development of gradient and recursive least-squares adaptive Volterra filters are first discussed. followed by adaptive algorithms using system models involving recursive nonlinear difference equations. Such systems are attractive because they may be able to approximate many nonlinear systems with great parsimony in the use pf coefficients. Also discussed are current research trends and new results and problem areas associated with these nonlinear filters. A lattice structure for polynomial models is also described

Introducing Adaptive filters Based on Shadow Concept for Speech Processing

This paper presents the new approach to introducing adaptive Filter with LMS Algorithm based on Shadow concept. Which is useful for the cancellation of the noise component overlap with Speech signal in the same frequency range, but fixed LMS algorithm produces minimum convergence rate and fixed steady state error. So we presents design, implementation and performance  of adaptive FIR filter, based on  Shadow concept, which produces minimum mean square error compare to fixed LMS, and we also obtains denoised Speech signal at output, and also we propose to calculate SNR values of  Adaptive Filter with LMS algorithm with and without Shadow concept

New sequential partial-update least mean M-estimate algorithms for robust adaptive system identification in impulsive noise

The sequential partial-update least mean square (S-LMS)-based algorithms are efficient methods for reducing the arithmetic complexity in adaptive system identification and other industrial informatics applications. They are also attractive in acoustic applications where long impulse responses are encountered. A limitation of these algorithms is their degraded performances in an impulsive noise environment. This paper proposes new robust counterparts for the S-LMS family based on M-estimation. The proposed sequential least mean M-estimate (S-LMM) family of algorithms employ nonlinearity to improve their robustness to impulsive noise. Another contribution of this paper is the presentation of a convergence performance analysis for the S-LMS/S-LMM family for Gaussian inputs and additive Gaussian or contaminated Gaussian noises. The analysis is important for engineers to understand the behaviors of these algorithms and to select appropriate parameters for practical realizations. The theoretical analyses reveal the advantages of input normalization and the M-estimation in combating impulsive noise. Computer simulations on system identification and joint active noise and acoustic echo cancellations in automobiles with double-talk are conducted to verify the theoretical results and the effectiveness of the proposed algorithms. © 2010 IEEE.published_or_final_versio

Enhanced Nonlinear System Identification by Interpolating Low-Rank Tensors

Function approximation from input and output data is one of the most investigated problems in signal processing. This problem has been tackled with various signal processing and machine learning methods. Although tensors have a rich history upon numerous disciplines, tensor-based estimation has recently become of particular interest in system identification. In this paper we focus on the problem of adaptive nonlinear system identification solved with interpolated tensor methods. We introduce three novel approaches where we combine the existing tensor-based estimation techniques with multidimensional linear interpolation. To keep the reduced complexity, we stick to the concept where the algorithms employ a Wiener or Hammerstein structure and the tensors are combined with the well-known LMS algorithm. The update of the tensor is based on a stochastic gradient decent concept. Moreover, an appropriate step size normalization for the update of the tensors and the LMS supports the convergence. Finally, in several experiments we show that the proposed algorithms almost always clearly outperform the state-of-the-art methods with lower or comparable complexity.Comment: 12 pages, 4 figures, 3 table