- Ce travail presente un modèle de comportement en régime permanant de l'algorithme «multi-split LMS». Des expressions de récurrences sont obtenues pour le vecteur de coefficients moyens et pour l'erreur carré moyenne. Des résultats de simulation démontrent une conformité excellent avec les prédictions théoriques et nous permettent de valider le modèle proposé

BELLANGER, M. G.

RESENDE, L. S.

SOUZA, R. D.

I-Revues

A Statistical Model for the Steady-State Behavior of the 
Multi-Split LMS Algorithm 
 
L. S. RESENDE1, R. D. SOUZA1 and M. G. BELLANGER2 
1Federal University of Santa Catarina, P. Box 476, 88040-900 Florianópolis-SC – BRAZIL, {leonardo,richard}@eel.ufsc.br 
2Conservatoire National des Arts et Métiers, 292 rue Saint-Martin, 75141 Paris Cedex 03 – FRANCE, bellang@cnam.fr 
Résumé – Ce travail presente un modèle de comportement en régime permanant de l’algorithme «multi-split LMS». Des expressions de 
récurrences sont obtenues pour le vecteur de coefficients moyens et pour l’erreur carré moyenne. Des résultats de simulation démontrent une 
conformité excellent avec les prédictions théoriques et nous permettent de valider le modèle proposé. 
Abstract – This paper presents a statistical model for the steady-state behavior of the multi-split LMS algorithm. Deterministic recursions 
are obtained for the mean weight vector and the mean square error. Simulation results display excellent agreement with the theoretical 
predictions and enable us to validate the proposed model. 
 
1. Introduction 
The multi-split processing technique has been used in 
adaptive systems for improving the convergence behavior of 
the LMS algorithm [1-3]. It consists of a continued splitting 
process of the filter impulse response in symmetric and 
antisymmetric parts. The filter is then realized as a set of 
zero-order filters connected in parallel, and with each single 
coefficient independently updated. Such a technique can be 
viewed as a transform domain filter, in which multi-split 
preprocessing is applied to the input data vector. 
An advantage of the multi-split transform is its ease of 
implementation. The computational burden is proportional to 
the number of filter coefficients N, and when N is equal to a 
power of two, the multi-split transform can be obtained by a 
butterfly computation scheme with no multiplication 
operation [1,4]. 
Recently, an analysis that justifies the improved 
performance of the multi-split LMS algorithm has been 
proposed in [4,5]. It is based on the fact that multi-split 
transform does not reduce the eigenvalue spread, but it does 
improve the diagonalization factor of the input signal 
correlation matrix, which is exploited by a power-
normalized, time-varying step-size LMS algorithm for 
updating the filter coefficients in adaptive systems. 
Our purpose in this paper is to start with a statistical 
analysis of the multi-split LMS algorithm and to present a 
model for the mean weight vector and the mean square error 
steady-state behavior. Deterministic recursions that predict 
such steady-state behaviors are derived, and their 
convergences towards the mean weight vector and the 
minimum mean-square error of the optimum filter are 
investigated. 
The paper is organized as follows. In Section 2 we briefly 
describe the multi-split Wiener filtering and present the 
multi-split LMS algorithm for adaptive systems. Section 3 is 
dedicated to the statistical analysis of such an algorithm, 
developing a model for the mean weight vector and the mean 
square error steady-state behavior. In Section 4 we present 
simulation results that validate our analysis. Finally, in 
Section 5 we draw our conclusions. 
2. Multi-split transversal filtering 
2.1 Optimum multi-split Wiener filter 
Consider initially the classical scheme of a nonadaptive 
transversal filter (Figure 1), where w denotes the N-by-1 tap-
weight vector and 
x(n)=[x(n), x(n−1), …, x(n−N+1)]t                  (1) 
the tap-input vector. The input signal x(n) and the desired 
response d(n) are modeled as wide-sense stationary discrete-
time stochastic processes of zero mean, Gaussian, with 
variance σx2 and σd2, respectively. The optimum weight 
vector wopt, called the Wiener vector, is given by [6-8] 
wopt=R
−1p,                                   (2) 
where R is the N-by-N correlation matrix of x(n), and p is the 
N-by-1 cross-correlation vector between x(n) and d(n). 
 
Figure 1: Transversal filtering. 
For ease of presentation, let N=2L, where L is an integer 
number greater than one. Without loss of generality, also 
consider that all the parameters are real-valued. 
It has been shown in [3,4] that the multi-split filtering 
problem can be formulated and solved by using a linearly-
constrained optimization, and can be implemented by means 
of a parallel GSC structure. The resulting multi-split filtering 
scheme can be represented by the block diagram in Figure 2, 
where 



−= 222
222
NNN
NNN
N MMJ
MJM
M ,                 (3) 
M1=[1] and w⊥i, for i=0, 1, …, N−1, are the single 
coefficients of the zero-order filters. It can be verified that M 
is a matrix of +1’s and –1’s, in which the inner product of 
any two distinct columns is zero. In fact, M is a nonsingular 
matrix and MtM=2LI. 
 Figure 2: Multi-split transform of the input x(n). 
The estimation error is then given by 
e(n)=d(n)−w⊥tx⊥(n),                           (4) 
where 
w⊥=[w⊥0, w⊥1, …, w⊥N−1]t                       (5) 
and 
x⊥(n)=Mtx(n)=[x⊥0(n), x⊥1(n), …, x⊥N−1(n)]t.        (6) 
In the mean-squared-error sense, w⊥ is chosen to minimize 
the following cost function: 
ξ(w⊥)=E{e2(n)}=σd2−2w⊥tMtp+w⊥tMtRMw⊥.        (7) 
The optimum solution is given by 
w⊥opt=[MtRM]−1Mtp=M−1R−1p=(1/2L)Mtwopt,        (8) 
and the scheme of Figure 2 corresponds to the optimum 
multi-split Wiener filter: 
wopt=Mw⊥opt.                                 (9) 
Substituting (8) in (7), the minimum mean-square error is 
found to be 
ξmin=σd2−ptR−1p=ptwopt=ptMw⊥opt,                (10) 
which is, therefore, equal to the minimum mean-square error 
of the optimum Wiener filter. 
2.2 Adaptive multi-split filtering 
It has been shown that the multi-split transform is not an 
input whitening transformation. Instead, it increases the 
diagonalization factor of the input signal correlation matrix 
without affecting its eigenvalue spread [4,5]. 
In the adaptive context, a power-normalized, time-varying 
step-size LMS algorithm, which exploits the nature of the 
transformed input correlation matrix, has been proposed for 
updating the single coefficients independently [3,4]. Table I 
presents a summary of the multi-split LMS algorithm. 
3. Steady-state algorithm behavior 
This section studies the limiting behavior of the converged 
multi-split LMS algorithm. To that end, let us assume that the 
variance estimates of x⊥i(n), for i=0, 1, …, N−1, have 
converged. Consequently, the step-sizes µ/ri(n) are 
considered fixed. However, they can still be distinct to each 
other, which is one of the bases of the multi-split LMS 
algorithm. 
3.1 Mean weight behavior 
Taking into account the aforementioned assumption, the 
weight update equation for the multi-split LMS algorithm can 
then be rewritten as follows: 
w⊥(n)=w⊥(n−1)+µΣ−1x⊥(n)e(n),                (11) 
Table I: Multi-Split LMS (MS-LMS) algorithm 
 
? Selection of parameters: 
adaptation step-size: 
222
1
0
x
Lσµ <<  
forgetting factor: 0<<γ ≤1 
? Initialization: 
for i=0,1, …, N−1, set: w⊥i(0)=0 and ri(0)=0 
set MN as such (3) 
? Updating: 
for i=0,1, …, N−1 and n=1,2, …, compute: 
1) Linear transform: x⊥i(n)=mitx(n), where mi is the 
(i+1)th column vector of MN 
2) Normalized LMS algorithm: 
∑−
=
⊥⊥ −=
1
0
)1()()(
N
i
ii nwnxny  
e(n)=d(n)−y(n) 
))1( )((
1
)1( )( 2 −−+−= ⊥ nrnxnnrnr iiii γγ  
)()(
)(
)1()( nenx
nr
nwnw i
i
ii ⊥⊥⊥ +−= µ  
 
 
where 
Σ=diag[σx⊥02, σ x⊥12, …, σ x⊥N−12]               (12) 
and 
e(n)=d(n)−w⊥t(n−1)x⊥(n),                    (13) 
The expected value of (11) leads to the recursion: 
        E{w⊥(n)}=E{w⊥(n−1)}+ 
                             +µΣ−1E{x⊥(n)d(n)}+ 
                             −µΣ−1E{x⊥(n)x⊥t(n)w⊥(n−1)} 
=[I−µΣ−1MtRM]E{w⊥(n−1)}+µΣ−1Mtp,   (14) 
where, for sufficiently small µ and assuming that x(n) and 
w⊥(n−1) are statistically independent, the following 
approximation has been used: 
       E{x⊥(n)x⊥t(n)w⊥(n−1)}≈E{x⊥(n)x⊥t(n)}E{w⊥(n−1)} 
=MtRM E{w⊥(n−1)}.            (15) 
Equation (14) is a deterministic recursion for the mean 
weight convergence of the multi-split LMS algorithm, in 
which its transient behavior with time-varying step-size has 
not been taken into account. 
3.1.1 Steady-state mean weight 
For the steady-state analysis, it is assumed that the 
algorithm converges as n→∞ and 
limn→∞w(n)=limn→∞E{w(n)}=w∞.                (16) 
Replacing w(n) with w∞ in (14) yields 
w∞=[MtRM]−1Mtp,                        (17) 
which corresponds to the optimum solution in (8). 
3.2 Mean square error behavior 
Squaring (13) and taking the expected value yields 
        ξ(n)=E{e2(n)}=σd2−2ptME{w⊥(n−1)}+ 
+tr[MtRME{w⊥(n−1)w⊥t(n−1)}],    (18) 
where the following approximation have been used: 
E{d(n)x⊥t(n)w⊥(n−1)}≈E{d(n)x⊥t(n)}E{w⊥(n−1)},  (19) 
         E{w⊥t(n−1)x⊥(n)x⊥t(n)w⊥(n−1)}= 
                          tr[E{x⊥(n)x⊥t(n)w⊥(n−1)w⊥t(n−1)}] 
≈tr[E{x⊥(n)x⊥t(n)}E{w⊥(n−1)w⊥t(n−1)}],  (20) 
and tr[?] stands for the trace of the matrix. 
3.2.1 Weight correlation matrix 
Evaluation of (18) requires E{w⊥(n−1)w⊥t(n−1)}. Post-
multiplying (11) by its transpose and taking the expected 
value yields 
K(n)=E{w⊥(n)w⊥t(n)} 
  =K(n−1)+µΣ−1MtpE{w⊥t(n−1)}−µΣ−1MtRMK(n−1)+ 
    +µE{w⊥(n−1)}ptMΣ−1−µK(n−1)MtRMΣ−1+ 
    +µ2Σ−1E{x⊥(n)d2(n)x⊥t(n)}Σ−1+ 
    −2µ2Σ−1E{x⊥(n)d(n)x⊥t(n)w⊥(n−1)x⊥t(n)}Σ−1+ 
    +µ2Σ−1E{x⊥(n)w⊥t(n−1)x⊥(n)x⊥t(n)w⊥(n−1)x⊥t(n)}Σ−1, (21) 
where the same approximation in (19) and (20) have been 
used. Now, since x(n) and d(n) are Gaussian, the last three 
expected values in (21) can be evaluated using the moment 
factoring theorem [8,9]. It can be shown that: 
E{x⊥(n)d2(n)x⊥t(n)}=2MtpptM+σd2MtRM,         (22) 
      E{x⊥(n)d(n)x⊥t(n)w⊥(n−1)x⊥t(n)}≈ 
          MtpE{w⊥t(n−1)}MtRM+MtRME{w⊥(n−1)}ptM+ 
+ptME{w⊥(n−1)}MtRM                                        (23) 
and 
      E{x⊥(n)w⊥t(n−1)x⊥(n)x⊥t(n)w⊥(n−1)x⊥t(n)}≈ 
                        E{w⊥t(n−1)}MtRME{w⊥(n−1)}MtRM+ 
+2MtRMK(n−1)MtRM. (24) 
Thus, (21) is a recursion for the weight correlation matrix 
and (18) for the mean square error behavior. As in (14), it is 
also worth pointing out that these deterministic expressions 
do not take into account the time-varying step-size. 
Consequently, they cannot predict accurately the transient 
behavior of the multi-split algorithm. 
3.2.2 Steady-state MSE 
An expression for the steady-state MSE behavior is 
determined by replacing w⊥(n−1) with the steady-state mean 
weight vector expression (17) in (18). It is given by 
limn→∞ξ(n)= σd2−ptR−1p=ξmin,                   (25) 
which corresponds to the minimum mean-square error in 
(10), as expected. 
4. Simulation results 
In order to validate the proposed analysis, we consider the 
same equalization system in [8, chap.5] (Figure 3). The input 
channel is binary, with b(n)=±1, and the impulse response of 
the channel is described by the raised cosine: 

 =−+=
π
otherwise   ,0
3 ,2 ,1   ))),2(cos(1( 22
1 jj
c Sj ,            (26) 
where S controls the eigenvalue spread χ(R) of the 
correlation matrix of the tap inputs in the equalizer, with 
χ(R)=6.0782 for S= 2.9 and χ(R)=46.8216 for S= 3.5. The 
sequence v(n) is an additive white noise that corrupts the 
channel output with variance σv2=0.001, and the equalizer 
has eleven coefficients. 
 
Figure 3: Adaptive equalizer for simulation. 
Figure 4 compares the simulated (100 independent trials) 
mean weight behavior with the analytical model using (14). 
The comparison between the simulated mean-square error 
behavior and the analytical model using (18) and (21) is 
shown in Figure 5. It can be verified that the proposed 
statistical analysis predicts with good accuracy the steady-
state behavior of the multi-split LMS algorithm, whereas the 
transient behavior has shown a lower convergence rate. It is 
due to the fact that the step-sizes have been considered fixed. 
The algorithm parameters were µ=0.0455 and γ=1. 
In fact, a statistical model that enables us to predict the 
transient behavior of the multi-split LMS algorithm must take 
into account the time-varying step-size aspect, which 
necessarily makes the analysis more complex. 
5. Conclusion 
This paper has presented a statistical analysis for the 
steady-state behavior of the multi-split LMS algorithm. 
Deterministic recursions for the mean weight vector and the 
mean square error have been derived. The convergence of 
such recursions towards the mean weight vector and the 
minimum mean-square error of the optimum filter has been 
analytically demonstrated and confirmed by simulations. 
Even though the analysis carried out in this paper is based on 
some of the results in [4,5], its outcomes validate the 
assumptions considered therein and contribute to the 
understanding of multi-split filtering. This kind of analysis is 
also useful for adaptive algorithm design and evaluation. 
Studies concerning to the development of a model for the 
transient behavior of the multi-split LMS algorithm are in 
progress. 
References 
[1] K.F. Wan and P.C. Ching, Multilevel split-path adaptive 
filtering and its unification with discrete Walsh 
transform adaptation. IEEE Transactions on Circuits 
and Systems-Part II; vol. 44; no. 2; pp. 147-151; 
February 1992. 
[2] R. D. Souza, L. S. Resende, C. A. F. da Rocha and M. 
G. Bellanger, On split FIR filtering in blind 
equalization. Proceedings of the IEEE ICC'02; New 
York; USA; April 2002. 
[3] L. S. Resende, J. M. T. Romano and M. G. Bellanger, 
Multi-split adaptive filtering. Proceedings of the IEEE 
ICASSP’01; Salt Lake City; USA; May 2001. 
[4] L. S. Resende, J. M. T. Romano and M. G. Bellanger, 
Split Wiener filtering with application in adaptive 
systems. Accepted for publication in IEEE Transactions 
on Signal Processing. 
[5] R. D. Souza, L. S. Resende and M. G. Bellanger, A 
Justification for the improved performance of the multi-
split LMS algorithm. Proceedings of the IEEE 
ICASSP’03; Hong Kong; May 2003. 
[6] B. Widrow and S.D. Stearns, Adaptive signal 
processing. New Jersey: Prentice-Hall, 1985. 
[7] M.G. Bellanger, Adaptive digital dilters and signal 
analysis, 2nd edition. New York and Basel: Marcel 
Dekker, Inc., 2001. 
[8] S. Haykin, Adaptive filter theory, 4th edition. New 
Jersey: Prentice-Hall, 2002. 
[9] A. Papoulis, Probability, randow variables and 
stochastic processes, 3rd edition. New York: McGraw-
Hill, 1991. 
0 100 200 300 400 500
0
0.05
0.1
0.15
0.2
Number of iterations (n)
F
ir
s
t 
th
re
e
 w
e
ig
h
ts
s imulated 
model 
 
(a): χ(R)=6.0782 
0 100 200 300 400 500
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
F
ir
s
t 
th
re
e
 w
e
ig
h
ts
Number of iterations (n)
s imulated 
model 
 
(b): χ(R)=46.8216 
Figure 4: Mean weight behavior. 
0 100 200 300 400 500
10
-3
10
-2
10
-1
10
0
10
1
Number of iterations (n)
m
e
a
n
 s
q
u
a
re
 e
rr
o
r
 
(a): χ(R)=6.0782 
0 100 200 300 400 500
10
-2
10
-1
10
0
10
1
Number of iterations (n)
m
e
a
n
 s
q
u
a
re
 e
rr
o
r
 
(b): χ(R)=46.8216 
Figure 5: Mean square error behavior. 


A statistical model for the steady-state behavior of the multi-split LMS algorithm

Abstract

Similar works

Full text

Available Versions

I-Revues