Maouche, Karim

English

EURECOM Repository

The Generalized Sliding Window RecursiveLeast-Squares (GSW RLS) AlgorithmKarim MAOUCHEyand Dirk T. M. SLOCKyyy;yyInstitut EURECOM, 2229 route des Cre^tesB.P. 193, 06904, Sophia Antipolis Cedex, FRANCEyTel/Fax: +33 93002632/ 93002627, E-Mail: maouche@eurecom.fryyTel/Fax: +33 93002606/ 93002627, E-Mail: slock@eurecom.frAbstract. In this paper, we derive a new RLS algorithm: the Generalized Sliding Window RLS (GSW RLS) algorithm that has abetter tracking ability than the SWC RLS algorithm. This algorithm uses a generalized window which consists of an exponential windowfor theL0most recent data and the same but attenuatedexponential window for the rest of the data. We give a theoritical proof that the useof this window leads to a better compromise between the Excess Mean Squared Errors due to estimation noise and lag noise. Furthermore,after providing a fast version of the GSW RLS algorithm, namely the GSW FTF algorithm, we apply the Subsampled-Upadating techniqueto derive the FSU GSW FTF algorithm, a doubly-fast version of the GSW RLS algorithm.1 IntroductionTracking ability is a desired feature in adaptive ltering, es-pecially when the system to identify can vary quickly as inthe case of acoustic echo channels. In the actual state of theart, two Finite Impulse Response (FIR) adaptive lteringalgorithms are known for their good tracking ability amongthe large set of FIR adaptive ltering algorithms: the AneProjection (AP) algorithm and the Recursive Least-Squaresalgorithm with a Sliding Rectangular Window (SWC RLS).The AP algorithm has a better tracking ability than theSWC RLS algorithm. It solves recursively an underdeter-mined system of linear equations while the SWC RLS al-gorithm solves recursively an overdetermined system of lin-ear equations. The SWC RLS algorithm exhibits a bettertracking ability than the classical RLS algorithm with anexponential window (WRLS). This is explained by the factthat the rectangular window allows to forget the past moreabruptly than the exponential window does.In this paper, we propose the GSW RLS algorithm, a newRLS algorithm that generalizes the WRLS and SWC RLSalgorithms and shows a better tracking ability than the SWCRLS algorithm. This algorithm uses a generalized windowwhich consists of the superposition of an exponential win-dow for the L0most recent data and the same but attenuatedexponential window for the rest of the data. The improve-ment of tracking comes from the fact that the length L0ofthe rst window can be much smaller than the length of therectangular window of the SWC RLS algorithm. Moreover,we prove theoretically that the use of this window leads toa better compromise between the Excess Mean Squared Er-rors (EMSE) due to estimation noise and lag noise. TheGSW RLS algorithm turns out to have the same structureas the SWC RLS algorithm. Its computational complexityis O(N2), the same as the SWC RLS algorithm. Never-theless, the exploitation of a certain shift-invariance prop-erty that is inherent to the adaptive ltering problem allowsthe derivation of fast version and leads to the GSW FastTransversal Filter (GSW FTF) algorithm whose computa-tional complexity is 14N (16N for the stabilized version),i.e., the same complexity as the SWC FTF algorithm. Fur-thermore, by applying the Subsampled-Updating techniqueto the GSW FTF algorithm, we derive a very fast versionof the GSW RLS algorithm called the FSU GSW RLS algo-rithm, a mathematically equivalent algorithm to the GSWRLS algorithm with a low computational complexity whendealing with long FIR lters.2 The GSW RLS AlgorithmAn adaptive transversal lterWN;kcombines linearly N con-secutive input samples fx(i n); n = 0; : : : ;N 1g to approx-imate (the negative of) the desired-response signal d(i). Theresulting error signal is given byN(ijk)=d(i) +WN;kXN(i)=d(i) +N 1Xn=0Wn+1N;kx(i n) ; (1)where XN(i) =xH(i) xH(i 1)   xH(i N+1)His the in-put data vector and superscriptHdenotes Hermitian (com-plex conjugate) transpose. In the WRLS algorithm, the setof N transversal lter coecients WN;k=W1N;k  WNN;kare adapted so as to minimize recursively the following LScriterionN(k) =kXi=1k ikN(ijk)k2; (2)where  2 (0; 1] is the exponential weighting factor, kvk2=vvH, k:k = k:kI. On the other hand, the SWC RLS algo-rithm minimizes recursively the following criterion:N;L0(k) =kXi=k L0+1kN(ijk)k2; (3)iL00(1 )L01Figure 1: The generalized window.where L0, the length of the sliding window, must be greaterthan the lter length: L0> N , in which case, the associatedcovariance matrix is invertible. Compared to the WRLS al-gorithm, the SWC RLS algorithm exhibits better tracking.In order to improve the tracking of the SWC RLS algorithm,one should use a rectangular window whose length is smallerthan the lter length. Doing this renders the problem singu-lar. Hence, in order to regularize the problem, we add to therectangular window of length L0, an attenuated exponentialwindow. This leads to minimizing the criterion over moredata and weighting the past data less heavily than the mostL0recent samples (see Fig. 1). Compared to the rectangu-lar window of the SWC RLS algorithm, the new windowintroduces two degrees of freedom, the attenuation and theforgetting factor of the exponential tail. Furthermore, it hasappeared that the derivation of a recursive algorithm for thenew window requires the use of an exponential window forthe L0most recent samples instead of the rectangular win-dow and with the same forgetting factor as the attenuatedexponential window. Hence, consider the following criterion:N;L0(k)=k L0Xi=1wk ikN(ijk)k2; wi=i0  i  L0(1 )ii > L0:(4)The new criterion generalizes the WRLS and SWC RLS cri-teria since the WRLS criterion (2) is obtained from (4) bysetting  = 0 and the SWC RLS criterion (4) is the onegiven by the generalized criterion when  = 1 and  = 1.Let WN;L0;kbe the adaptive RLS lter provided by such awindow, the minimization of (4) leads to the followingWN;L0;k=  PHN;L0;kR 1N;L0;k; (5)whereRN;L0;k= (1 )k L0Xi=1k iXN(i)XHN(i) +kXi=k L0+1k iXN(i)XHN(i)PN;L0;k= (1 )k L0Xi=1k iXN(i)dH(i) +kXi=k L0k iXN(i)dH(i) :(6)The use of the same forgetting factor for the two windowsallows the following recursions for the sample second ordermomentsRN;L0+1;k= RN;L0;k 1+XN(k)XHN(k)= RN;L0;k  L0XN(k L0)XHN(k L0)(7)PN;L0+1;k= PN;L0;k 1+XN(k)dH(k)= PN;L0;k L0XN(k L0)dH(k L0) :(8)Hence, we can derive the new algorithm by applying thestrategy for the usual WRLS algorithm twice. The rst stepwill be devoted to the time and order update (k 1; L0) !(k;L0+1), which is analogous to the update of the usualWRLS algorithm while the second step will be the orderdowndate (k;L0+1) ! (k;L0). The downdate scheme is ob-tained as follows: By using (8), one hasWN;L0;kRN;L0;k=WN;L0+1;kRN;L0+1;k L0d(k L0)XHN(k L0): (9)Using (7) for RN;L0+1;kin term of RN;L0;k, we getWN;L0;k=WN;L0+1;k+ L0N;L0+1(k)eDN;L0+1;k; (10)where N;L0+1(k) = d(k L0) + WN;L0+1;kXN(k L0) andeDN;L0+1;k=  XHN(k L0)R 1N;L0;kare the a posteriori errorsignal and the a priori Kalman gain of the downdate part.Applying the MIL to (7) givesR 1N;L0;k=R 1N;L0+1;k DHN;L0+1;k 1N;L0+1(k)DN;L0+1;k; (11)with DN;L0+1;k=  XHN(k L0)R 1N;L0+1;kand N;L0+1(k) = 1 L0  DN;L0+1;kXN(k L0) are respectively the a pos-teriori Kalman gain and the likelihood variable associatedwith the downdate part. Now, it is straigthtforward to ndthateDN;L0+1;k=  1 L0 1N;L0+1(k)DN;L0+1;kand that thea priori error is pN;L0+1(k) = d(k L0)+WN;L0;kXN(k L0) = 1 L0 1N;L0+1(k)N;L0+1(k). By associating the updatepart to the downdate part, we nd the GSW RLS algorithmeCN;L0;k=   1XHN(k)R 1N;L0;k 1 1N;L0(k) = 1 eCN;L0;kXN(k)pN;L0(k) = d(k) +WN;L0;k 1XN(k)N;L0(k) = pN;L0(k)N;L0(k)WN;L0+1;k= WN;L0;k 1+eCN;L0;kN;L0(k)R 1N;L0+1;k=  1R 1N;L0;k 1 eCHN;L0;kN;L0(k)eCN;L0;k(12)DN;L0+1;k=  XHN(k L0)R 1N;L0+1;kN;L0+1(k) =  1 L0 DN;L0+1;kXN(k L0)N;L0+1(k) = d(k L0) +WN;L0+1;kXN(k L0)pN;L0+1(k) =  1 L0 1N;L0+1(k)N;L0+1(k)WN;L0;k= WN;L0+1;k+ L0DN;L0+1;kpN;L0+1(k)R 1N;L0k= R 1N;L0+1;k DHN;L0+1;k 1N;L0+1(k)DN;L0+1;k:The set of equations (12) constitute the complete time up-date of the algorithm. The algorithm is initialized withRN;L0;0= I where  is a small scalar quantity. The GSWRLS algorithm shows a computational complexity of O(N2).One must notice that the GSW RLS algorithm has the samestructure and complexity as the SWC RLS algorithm. More-over, we prove in the next section that the use of this windowleads to a better compromise between the EMSE due to es-timation noise and lag noise.3 Performance analysisFor the purpose of analysis, we consider the following clas-sical identication model for the desired signald(k) =WoN;k 1XN(k) + n(k) ; (13)where n(k) is a centered Gaussian i.i.d. sequence with vari-ance 2n n(k)  N (0; 2n)and WoN;kis the unknown lterthat is time-varying according to a random walkWoN;k= WoN;k 1+ Z(k) ; Z(k) i.i.d.  N (0;Q) : (14)Considering a general window whose coecients are denotedby wi, one can show that the deviation lterfWN;L0;k=WN;L0;k+WoN;kis given byfWN;L0;k= 1Xi=0wi 1Xj=k iZ(j)!XN(k i)XN(k i)H 1Xi=0win(k i)XN(k i)!R 1N;L0;k:Let COVkbe the covariance matrix of the deviation l-ter: COVk= EfWN;L0;kfWHN;L0;k. Using the fact that E(:) =EXEn;ZjX(:) and assuming that k is big enough so that:RN;L0;k ERN;L0;k= 1Xi=0wi!R, we get after some ma-nipulationsCOVk= 2n 1Xi=0ew2i!R 1+ 1Xi=0w2i!Q ; (15)whereewi= wi 1Xj=0wj! 1; wi= 1 i 1Xj=0ewj(16)We see from (15) that there are two contributions in COVk:the rst one is the contribution related to estimation noisewhile the second one is for lag noise and originates fromthe variation of the echo path response. Now, assumingstatistical independence betweenfWN;L0;kand XN(k 1), wecan express the variance of the a priori error signal asvar(pN(k)) = 2n+ tr (RCOVk 1) (17)= 2n+N 1Xj=0ew2j!2+ 1Xj=0w2j!tr (RQ) :Hence, we can compute the EMSE due to estimation noisefor the WRLS, SWC RLS and GSW RLS algorithms:GSWRLS : 2nN1 1+1+( 2)2L0(1 L0)2WRLS : 2nN1 1+SWCRLS : 2nNL0:(18)The EMSE of WRLS can be recovered from the EMSE ofthe GSW RLS by letting  = 0. The same thing holds forthe SWC RLS algorithm for  = 1 and  ! 1. TakingR = 2xI and Q = 2zI, leads to the following results for theEMSE due to lag noiseGSWRLS : N2x2z1 2L0(1+ L0+1)+22L0(1+L0 L02)(1 2)(1 L0)2WRLS : N2x2z11 2SWCRLS : N2x2z(L0+1)(2L0+1)6L0:(19)Figure (2), shows curves that give lag noise misadjustment0.4 0.5 0.6 0.7 0.8 0.9 10.20.30.40.50.60.70.80.9Noise misadjustmentLag misadjustmentGSW RLSSWC RLSFigure 2: Comparison of SWC RLS and GSW RLSalgorithms (N = 50; 2x= 2z= :01; 2n= :1).vs. estimation noise misadjustment for N = 50; 2x= 2z=:01 and 2n= :1. These curves show that for the same valueof the estimation noise misadjustment, the GSW RLS algo-rithm has a lower lag noise misadjustment than the SWCRLS algorithm, so we can conclude that the GSW RLSalgorithm is truly better than the SWC RLS algorithm.Exploitation of the shift-invariance property leads to thederivation of a fast version of the GSW RLS algorithm,namely the GSW FTF algorithm.4 The GSW FTF algorithmThe GSW FTF algorithm can be described in the followingway, which emphasizes its rotational structure (see [1]) :266664heCN;L0;k0iAN;L0;kBN;L0;kDN;L0p;k0[WN;L0;k0]377775= k266664h0eCN;L0;k 1iAN;L0;k 1BN;L0;k 10 DN;L0p;k 1[WN;L0;k 10]377775epN;L0(k) = AN;L0;k 1XNp(k)eN;L0p(k) = epN;L0(k)N;L0(k 1) 1Np;L0(k) =  1N;L0(k 1) eC0Np;L0;kepN;L0(k) 1N;L0(k) =  1Np;L0(k)+eCNNp;L0;krpN;L0(k)rpN(k) =  N(k 1)eCN HNp;L0;k 1N;L0p(k) = 1 1N;L0(k 1) eC0HNp;L0;kNp;L0(k)eC0Np;L0;krN;L0p(k) = rpN;L0(k)N;L0(k)N;L0p(k) = N;L0(k 1) + rpN;L0(k) rHN;L0p(k)aN;L0p(k) = AN;L0p;kXNp(k L0+1) (20)asN;L0(k) = aN;L0p(k) 1N;L0p(k 1)N;L0(k) = N;L0p(k)  L0asN;L0 (k)asHN;L0 (k)Np;L0(k) = N;L0(k 1) D0Np;L0p;kaN;L0p(k)bN;L0p(k) =  N;L0p(k)DNHNp;L0p;kN;L0(k) = Np;L0(k) +DNNp;L0p;kbN;L0p(k)bsN;L0(k) = bN;L0p(k) 1N;L0p(k)N;L0(k) = N;L0p(k)  L0bsN;L0 (k)bsHN;L0 (k) ;where L0p= L0+1 and Np= N+1. AN;L0;kand BN;L0;karethe forward and backward prediction lters that are used inthe update part, epN;L0(k) and eN;L0(k) are the a priori anda posteriori forward prediction errors, rpN;L0(k) and rN;L0(k)are the a priori and a posteriori backward predition errors,eCNp;k=heC0Np;k  eCNNp;kiand N;L0(k) and N;L0(k) arethe forward and backward prediction error variances. kisa 5 5 rotation matrix given byk= 6k5k4k3k2k1k; (21)where the 5  5 matrices ik; i = 1; 2; 3; 4; 5; 6 are1k=266641 a 0 0 0b 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 1377752k=266641 0 c 0 00 1 0 0 00 0 1 0 00 0 0 1 00 0 0 0 1377753k=266641 0 0 0 00 1 0 0 0d 0 1 0 00 0 0 1 0e 0 0 0 1377754k=266641 0 0 0 00 1 0 f 00 0 1 0 00 g 0 1 00 0 0 0 137775(22)5k=266641 0 0 0 00 1 0 0 00 0 1 0 00 0 h 1 00 0 0 0 1377756k=266641 0 0 0 00 1 0 0 00 0 1 i 00 0 0 1 00 0 0 j 137775;with a =  epHN;L0(k) 1N;L0(k) , b = eN;L0p(k) , c =  eCNN;L0;k, d = rN;L0p(k) , e = N;L0(k) , f =  asN;L0p(k) , g = aHN;L0p(k) 1N;L0p(k) , h =  DNN;L0p;k, i =  bsN;L0(k) andj = L0N;L0p(k). The computational complexity of theGSW FTF algorithm is 14N operations, which is also thecomplexity of the SWC FTF algorithm. A numerically sta-bilized version of the GSW FTF algorithm is given in [2]and has a complexity of 16N operations.5 The FSU GSW FTF AlgorithmIn [3], [4] and [5], we have pursued an alternative way toreduce the complexity of adaptive ltering algorithms. Theapproach consists of subsampling the lter adaptation, i.e.the LS lter estimate is no longer provided every sample butevery L  1 samples (subsampling factor L). This strategyhas led us to derive new algorithms that are the FSU FTF,FSU FNTF and FSU FAP algorithms which present a re-duced complexity when dealing with long lters. Here, weapply this technique to the GSW FTF algorithm. Using thelter estimates at a certain time instant, we compute thelter outputs over the next L time instants. Using whatwe have called a GSW FTF-Schur algorithm, it is possi-ble to compute the successive rotation matrices of the GSWFTF algorithm for the next L time instants. Applying theL rotation matrices to the lters vectors becomes an issueof multiplying polynomials, which can be eciently carriedout using the FFT. The complexity of the FSU GSW FTF isO((10N+1L+26)FFT (2L)L+50NL+25L) operations per sample.This can be very interesting for long lters. For example,when (N;L) = (4095; 256); (8191; 256) and the FFT is donevia the split radix (FFT (2m) =mlog2(2m) real multiplica-tions for real signals) the multiplicative complexity is respec-tively 2:2N and 1:4N per sample. This should be comparedto 14N for the GSW FTF algorithm and 2N for the LMSalgorithm. The number of additions is somewhat higher.The cost we pay is a processing delay which is of the orderof L samples. The subsampled updating technique turnsout to be especially applicable in the case of very long l-ters such as occur in the acoustic echo cancellation problem.The computational gain it oers is obtained in exchange forsome processing delay, as is typical of block processing.6 Concluding RemarksOne major drawback of the SWC RLS algorithm is noiseamplication that is due to the fact that the estimation ofthe covariance matrix of order N is done using a rectangularwindow of a relatively short length L0(L0> N). With sucha window, the covariance matrix can be ill- conditionned ifthe input signal is not active in a signicant portion of thewindow. The WRLS algorithm is less sensitive to noise am-plication because of the larger memory of the exponentialwindow. Due to its exponential tail, the generalized windowreduces noise amplication. The AP algorithm also suersfrom noise amplication and the use of the generalized win-dow with the AP algorithm is the object of ongoing research.REFERENCES[1] D.T.M. Slock and T. Kailath \A modular prewindowingframework for covariance FTF RLS algorithms". SignalProcessing, 28(1):47{61, 1992.[2] K. Maouche and Dirk T.M. Slock. \The GeneralizedSliding Window Recursive Least-Squares (GSW RLS)Algorithm". Technical Report RR No95-021, InstitutEurecom, Sophia Antipolis, France, February 1995.[3] D.T.M. Slock and K. Maouche. \The Fast Subsampled-Updating Fast Transversal Filter (FSU FTF) RLS Algo-rithm". Annals of Telecommunications, 49(7-8):407{413,1994.[4] K. Maouche and D.T.M. Slock. \The Fast Subsampled-Updating Fast Newton Transversal Filter (FSU FNTF)for Adapting Long FIR Filters". In Proc. 28th AsilomarConf. on Signals, Systems and Computers, pages 1488{1492, Pacic Grove, CA, Oct. 31 - Nov. 2 1994.[5] K. Maouche and Dirk T.M. Slock. \The FastSubsampled-Updating Fast Ane Projection (FSUFAP) Algorithm". Technical Report RR No95-018, In-stitut Eurecom, Sophia Antipolis, France, January 1995.

The generalized sliding window recursive least-square (GSW RLS)

http://www.eurecom.fr/en/publication/24/download/cm-maouka-950621.pdf

The generalized sliding window recursive least-square (GSW RLS)

Abstract

Similar works

Full text

Available Versions

EURECOM Repository