Collaborative adaptive filtering for machine learning

Quantitative performance criteria for the analysis of machine learning architectures and algorithms have long been established. However, qualitative performance criteria, which identify fundamental signal properties and ensure any processing preserves the desired properties, are still emerging. In many cases, whilst offline statistical tests exist such as assessment of nonlinearity or stochasticity, online tests which not only characterise but also track changes in the nature of the signal are lacking. To that end, by employing recent developments in signal characterisation, criteria are derived for the assessment of the changes in the nature of the processed signal. Through the fusion of the outputs of adaptive filters a single collaborative hybrid filter is produced. By tracking the dynamics of the mixing parameter of this filter, rather than the actual filter performance, a clear indication as to the current nature of the signal is given. Implementations of the proposed method show that it is possible to quantify the degree of nonlinearity within both real- and complex-valued data. This is then extended (in the real domain) from dealing with nonlinearity in general, to a more specific example, namely sparsity. Extensions of adaptive filters from the real to the complex domain are non-trivial and the differences between the statistics in the real and complex domains need to be taken into account. In terms of signal characteristics, nonlinearity can be both split- and fully-complex and complex-valued data can be considered circular or noncircular. Furthermore, by combining the information obtained from hybrid filters of different natures it is possible to use this method to gain a more complete understanding of the nature of the nonlinearity within a signal. This also paves the way for building multidimensional feature spaces and their application in data/information fusion. To produce online tests for sparsity, adaptive filters for sparse environments are investigated and a unifying framework for the derivation of proportionate normalised least mean square (PNLMS) algorithms is presented. This is then extended to derive variants with an adaptive step-size. In order to create an online test for noncircularity, a study of widely linear autoregressive modelling is presented, from which a proof of the convergence of the test for noncircularity can be given. Applications of this method are illustrated on examples such as biomedical signals, speech and wind data

Sparseness-controlled adaptive algorithms for supervised and unsupervised system identification

In single-channel hands-free telephony, the acoustic coupling between the loudspeaker and the microphone can be strong and this generates echoes that can degrade user experience. Therefore, effective acoustic echo cancellation (AEC) is necessary to maintain a stable system and hence improve the perceived voice quality of a call. Traditionally, adaptive filters have been deployed in acoustic echo cancellers to estimate the acoustic impulse responses (AIRs) using adaptive algorithms. The performances of a range of well-known algorithms are studied in the context of both AEC and network echo cancellation (NEC). It presents insights into their tracking performances under both time-invariant and time-varying system conditions. In the context of AEC, the level of sparseness in AIRs can vary greatly in a mobile environment. When the response is strongly sparse, convergence of conventional approaches is poor. Drawing on techniques originally developed for NEC, a class of time-domain and a frequency-domain AEC algorithms are proposed that can not only work well in both sparse and dispersive circumstances, but also adapt dynamically to the level of sparseness using a new sparseness-controlled approach. As it will be shown later that the early part of the acoustic echo path is sparse while the late reverberant part of the acoustic path is dispersive, a novel approach to an adaptive filter structure that consists of two time-domain partition blocks is proposed such that different adaptive algorithms can be used for each part. By properly controlling the mixing parameter for the partitioned blocks separately, where the block lengths are controlled adaptively, the proposed partitioned block algorithm works well in both sparse and dispersive time-varying circumstances. A new insight into an analysis on the tracking performance of improved proportionate NLMS (IPNLMS) is presented by deriving the expression for the mean-square error. By employing the framework for both sparse and dispersive time-varying echo paths, this work validates the analytic results in practical simulations for AEC. The time-domain second-order statistic based blind SIMO identification algorithms, which exploit the cross relation method, are investigated and then a technique with proportionate step-size control for both sparse and dispersive system identification is also developed

Adaptive filters for sparse system identification

Sparse system identification has attracted much attention in the field of adaptive algorithms, and the adaptive filters for sparse system identification are studied. Firstly, a new family of proportionate normalized least mean square (PNLMS) adaptive algorithms that improve the performance of identifying block-sparse systems is proposed. The main proposed algorithm, called block-sparse PNLMS (BS-PNLMS), is based on the optimization of a mixed ℓ2,1 norm of the adaptive filter\u27s coefficients. A block-sparse improved PNLMS (BS-IPNLMS) is also derived for both sparse and dispersive impulse responses. Meanwhile, the proposed block-sparse proportionate idea has been extended to both the proportionate affine projection algorithm (PAPA) and the proportionate affine projection sign algorithm (PAPSA). Secondly, a generalized scheme for a family of proportionate algorithms is also presented based on convex optimization. Then a novel low-complexity reweighted PAPA is derived from this generalized scheme which could achieve both better performance and lower complexity than previous ones. The sparseness of the channel is taken into account to improve the performance for dispersive system identification. Meanwhile, the memory of the filter\u27s coefficients is combined with row action projections (RAP) to significantly reduce the computational complexity. Finally, two variable step-size zero-point attracting projection (VSS-ZAP) algorithms for sparse system identification are proposed. The proposed VSS-ZAPs are based on the approximations of the difference between the sparseness measure of current filter coefficients and the real channel, which could gain lower steady-state misalignment and also track the change in the sparse system --Abstract, page iv

Algoritmos adaptativos LMS normalizados proporcionais: proposta de um novo algoritmo e sua modelagem estocástica

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-Graduação em Engenharia Elétrica.Neste trabalho, um novo algoritmo LMS normalizado proporcional (PNLMS) é proposto. Tal algoritmo usa fatores de ativação individuais para cada coeficiente do filtro adaptativo, em vez de um fator de ativação global como no algoritmo PNLMS padrão. Os fatores de ativação individuais do algoritmo proposto são atualizados recursivamente a partir dos correspondentes coeficientes do filtro adaptativo. Essa abordagem conduz a uma melhor distribuição da energia de adaptação entre os coeficientes do filtro. Dessa forma, para respostas ao impulso com elevada esparsidade, o algoritmo proposto, denominado algoritmo PNLMS com fatores de ativação individuais (IAF PNLMS), atinge maior velocidade de convergência do que os algoritmos PNLMS padrão e PNLMS melhorado (IPNLMS). Também, uma metodologia de modelagem estocástica dos algoritmos da classe PNLMS é apresentada. Usando essa metodologia, obtém-se um modelo estocástico que prediz satisfatoriamente o comportamento do algoritmo IAF PNLMS tanto na fase transitória quanto na estacionária. Através de simulações numéricas, a eficácia do modelo proposto é verificada. Adicionalmente, uma versão melhorada do algoritmo IAF PNLMS, denominada EIAF PNLMS, é proposta neste trabalho, a qual usa uma estratégia de redistribuição de ganhos durante o processo de aprendizagem, visando aumentar os ganhos atribuídos aos coeficientes inativos quando os ativos aproximam-se da convergência. Resultados de simulação mostram que tal estratégia de redistribuição melhora significativamente as características de convergência do algoritm

Adaptive Filtered-x Algorithms for Room Equalization Based on Block-Based Combination Schemes

(c) 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works.[EN] Room equalization has become essential for sound reproduction systems to provide the listener with the desired acoustical sensation. Recently, adaptive filters have been proposed as an effective tool in the core of these systems. In this context, this paper introduces different novel schemes based on the combination of adaptive filters idea: a versatile and flexible approach that permits obtaining adaptive schemes combining the capabilities of several independent adaptive filters. In this way, we have investigated the advantages of a scheme called combination of block-based adaptive filters which allows a blockwise combination splitting the adaptive filters into nonoverlapping blocks. This idea was previously applied to the plant identification problem, but has to be properly modified to obtain a suitable behavior in the equalization application. Moreover, we propose a scheme with the aim of further improving the equalization performance using the a priori knowledge of the energy distribution of the optimal inverse filter, where the block filters are chosen to fit with the coefficients energy distribution. Furthermore, the biased block-based filter is also introduced as a particular case of the combination scheme, especially suited for low signal-to-noise ratios (SNRs) or sparse scenarios. Although the combined schemes can be employed with any kind of adaptive filter, we employ the filtered-x improved proportionate normalized least mean square algorithm as basis of the proposed algorithms, allowing to introduce a novel combination scheme based on partitioned block schemes where different blocks of the adaptive filter use different parameter settings. Several experiments are included to evaluate the proposed algorithms in terms of convergence speed and steady-state behavior for different degrees of sparseness and SNRs.The work of L. A. Azpicueta-Ruiz was supported in part by the Comtmidad de Madrid through CASI-CAM-CM under Grant S2013/ICE-2845, in part by the Spanish Ministry of Economy and Competitiveness through DAMA under Grant TIN2015-70308-REDT, and Grant TEC2014-52289-R, and in part by the European Union. The work of L. Fuster, M. Ferrer, and M. de Diego was supported in part by EU together with the Spanish Government under Grant TEC2015-67387-C4-1-R (MINECO/FEDER), and in part by the Cieneralitat Valenciana under Grant PROMETEOII/2014/003. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Simon Dodo.Fuster Criado, L.; Diego Antón, MD.; Azpicueta-Ruiz, LA.; Ferrer Contreras, M. (2016). Adaptive Filtered-x Algorithms for Room Equalization Based on Block-Based Combination Schemes. IEEE/ACM Transactions on Audio, Speech and Language Processing. 24(10):1732-1745. https://doi.org/10.1109/TASLP.2016.2583065S17321745241

Contribuições sobre algoritmos adaptativos LMS normalizados proporcionais

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia Elétrica, Florianópolis, 2015.Este trabalho de pesquisa apresenta uma nova política de distribuição de ganho para algoritmos tipo proporcional baseada na convergência individual dos coeficientes. Para isso, uma taxa de variação suavizada e normalizada da magnitude do coeficiente é concebida para avaliação de convergência individual dos coeficientes. A nova abordagem visa melhorar a distribuição de ganho durante o processo adaptativo. Para tal, ganhos associados a coeficientes ativos que estão na vizinhança de seus valores ótimos são reduzidos e redistribuídos a outros coeficientes visando, assim, acelerar a velocidade de convergência global do algoritmo. A partir da nova política de distribuição de ganho, três novas versões de algoritmos tipo proporcional são derivadas. Além disso, uma nova versão do algoritmo adaptativo proporcional ao desvio quadrático médio dos coeficientes (z2 proportionate) é apresentada. Este último algoritmo combina uma distribuição de ganho proporcional com ganho uniforme. Tal estratégia é dependente do conhecimento do nível de potência do ruído de medição presente no sistema que, na prática, não está sempre disponível. Assim, para contornar essa dependência, um novo procedimento de distribuição de ganho baseado na autocorrelação do sinal de erro é apresentado e discutido. O novo algoritmo supera o algoritmo original em termos de velocidade de convergência e resposta a perturbações na planta. Por fim, uma nova política de distribuição de ganho para algoritmos tipo proporcional para operação em ambientes com elevada esparsidade é proposta. A nova política utiliza uma função de amplificação do ganho de coeficientes ativos visando aumentar sua velocidade de convergência. A partir da nova política, dois novos algoritmos para operação com plantas cujas respostas ao impulso exibem elevada esparsidade são introduzidos. Resultados de simulação corroboram a eficácia dos algoritmos propostos.Abstract : This research work presents a new gain distribution policy for proportionate-type algorithms based on individual-coefficient convergence. To this end, a normalized and smoothed variation rate of the individual-coefficient magnitude is derived in order to assess the individual-coefficient convergence. The new approach aims to enhance the gain distribution during the adaptation process. Thereby, gains of the active coefficients that are close to their optimum values are reduced and redistributed to other coefficients, increasing the convergence speed of the algorithm. By using this policy, three new versions of proportionate algorithms have been conceived. Moreover, an alternative version of the mean-square weight deviation-proportionate gain algorithm (z2 proportionate) is introduced. This latter algorithm applies a rule combining the mean-square weight deviation-proportionate gain and a uniform gain to obtain the whole algorithm gain distribution. Such a rule is strongly dependent on the knowledge of the measurement noise variance, requiring therefore its estimate. Thereby, a novel approach aiming to circumvent such a dependence, based on error autocorrelation, is presented and discussed. Lastly, a new proportionate gain distribution strategy for operating with plants exhibiting high sparseness is proposed. The new policy uses an amplification function of the gain assigned to active coefficients in order to increase their convergence rate. Thereby, two proportionate algorithms have been developed. Through numerical simulation results, the effectiveness of the proposed algorithms is verified

Adaptive Algorithms for Intelligent Acoustic Interfaces

Modern speech communications are evolving towards a new direction which involves users in a more perceptive way. That is the immersive experience, which may be considered as the “last-mile” problem of telecommunications. One of the main feature of immersive communications is the distant-talking, i.e. the hands-free (in the broad sense) speech communications without bodyworn or tethered microphones that takes place in a multisource environment where interfering signals may degrade the communication quality and the intelligibility of the desired speech source. In order to preserve speech quality intelligent acoustic interfaces may be used. An intelligent acoustic interface may comprise multiple microphones and loudspeakers and its peculiarity is to model the acoustic channel in order to adapt to user requirements and to environment conditions. This is the reason why intelligent acoustic interfaces are based on adaptive filtering algorithms. The acoustic path modelling entails a set of problems which have to be taken into account in designing an adaptive filtering algorithm. Such problems may be basically generated by a linear or a nonlinear process and can be tackled respectively by linear or nonlinear adaptive algorithms. In this work we consider such modelling problems and we propose novel effective adaptive algorithms that allow acoustic interfaces to be robust against any interfering signals, thus preserving the perceived quality of desired speech signals. As regards linear adaptive algorithms, a class of adaptive filters based on the sparse nature of the acoustic impulse response has been recently proposed. We adopt such class of adaptive filters, named proportionate adaptive filters, and derive a general framework from which it is possible to derive any linear adaptive algorithm. Using such framework we also propose some efficient proportionate adaptive algorithms, expressly designed to tackle problems of a linear nature. On the other side, in order to address problems deriving from a nonlinear process, we propose a novel filtering model which performs a nonlinear transformations by means of functional links. Using such nonlinear model, we propose functional link adaptive filters which provide an efficient solution to the modelling of a nonlinear acoustic channel. Finally, we introduce robust filtering architectures based on adaptive combinations of filters that allow acoustic interfaces to more effectively adapt to environment conditions, thus providing a powerful mean to immersive speech communications