The problem of blind source separation in additive white noise is an important problem in speech, array and acoustic signal processing. In general this problem requires the use of higher order statistics of the received signals. However for many signal sources, such as speech with distinct non-white power spectral densities, second order statistics of the received signal mixture can be exploited for signal separation. While previous approaches often assume that additive noise is absent or that the noise correlation matrix is known, we propose a simple and yet effective signal extraction method for signal source separation under unknown white noise. This new and unbiased signal extractor is derived from the matrix pencil formed between output auto-correlation matrices at different delays. Simulation examples are presented.published_or_final_versio

Chan, FHY

Chang, C

Ding, Z

Yau, SF

The problem of blind source separation in additive white noise is an important problem in speech, array and acoustic signal processing. In general this problem requires the use of higher order statistics of the received signals. Nonetheless, many signal sources such as speech with distinct, non-white power spectral densities, second order statistics of the received signal mixture can be exploited for signal separation. While previous approaches often assume that additive noise is absent or that the noise correlation matrix is known, we propose a simple and yet effective signal extraction method for signal source separation under unknown white noise. This new and unbiased signal extractor is derived from the matrix pencil formed between output auto-correlation matrices at different delays. Simulation examples are presented.link_to_subscribed_fulltex

Ding, Zhi

Chang, Chunqi

Yau, Sze Fong

Chan, Francis HY

HKU Scholars Hub

Matrix-pencil approach to blind separation of non-white sources in white noise

Title A matrix-pencil approach to blind separation of non-whitesourcesin white noiseAuthor(s) Chang, C; Ding, Z; Yau, SF; Chan, FHYCitationIEEE International Conference on Acoustics, Speech and SignalProcessing Proceedings, Seattle, WA, USA, 12-15 May 1998, v. 4,p. 2485-2488Issued Date 1998URL http://hdl.handle.net/10722/46052Rights Creative Commons: Attribution 3.0 Hong Kong LicenseA MATRIX-PENCIL APPROACH TO BLIND SEPARATIOIN OF NON-WHITE SOURCES IN WHITE NOISE Chunqi Chang', Zhi Ding', Sze Fong Yau' and Francis H Y Chan' 'Department of Electrical and Electronic Engineering The University of Hong Kong, Pokfulam Road, Hong Kong 'Department of Electxical and Electrical Engineering Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong A B S T R A C T  The problem of blind source separation in additive white noise is an important problem in speech, array and acoustic signal processing. In general this problem requires the use of higher order statistics of the received signals. Nonethe- less, many signal sources such as speech with distinct, non-white power spectral densities, second order statistics of the received signal mixture can be exploited for signal separation. While previous approaches often assume that additive noise is absent or that the noise correlation ma- trix is known, we propose a simple and yet effective signal extraction method for signal source separation under im- known white noise. This new and unbiased signal extractor is derived from the matrix pencil formed between output auto-correlation matrices at different delays. Simulation examples are presented. 1. I N T R O D U C T I O N  Blind source separation has become a well established re- search topic in the signal processing community. It is moti- vated by practical scenarios with involve multi-sources and multi-sensors. The key objective of blind source separation is to extract the source signals from the sensor measure- ments without the knowledge of the characteristic of the transmission channel. Examples include antenna beam- forming, multi-person speech separation (cocktail party problem), and multi-channel medical signal separation. Many papers concerned with this topic have been pub- lished, both on separation principles and specific algo- rithms. However, the most commonly used assumptions are (1) source signals are white and mutually independent (2) a t  most one source is Gaussian (3) multiple sensor out- puts are lsnearly independent. Based on these assump- tions, separation principle have been investigated [1][2], and algorithms exploiting higher order statistics have been proposed[3]. Recently second order statistics based algorithms are presented under the assumption that the sources for sepa- ration are colored signals [4][5][6]. Second order statistics based method does not rely on the the non-Gaussian as- sumption. They may generate better performance than algorithms based on higher order statistics. For practical applications involving separations of speech and music :jig- nals which are typically non-white, second order statistical methods can be used. However, existing second order statistical methods of- ten assume the absence of additive noise at sensors. To overcome this critical weakness, we propose a second or- der statistics based matrix pencil approach in this paper. The new approach yields unbiased signal estimates from signal mixtures corrupted by additive white noise. This paper is organized as follows. In section 2, the problem of blind source separation is described along with relevant assumptions on second order statistics of signals for sepa- ration. Section 3 outlines a basic signal separability based on second order statistics. In section 4, a new matrix pen- cil method for blind souirce separation is presented. Monte Carlo simulations of the proposed method are presented in section 5. 2. PROBLEM FORMULATION A memoryless mixture of multiple signals is modeled by the following equation Z(n) := As'(n) + w'(n), (1) in which ?(n) is a vector of source signals of dimension N and Z(n) is the received signal vector of dimension M .  In (l), G ( n )  is the additive white noise vector and A is a M x N memoryless mixing matrix. In order for all sourl:es to be separable, A must have full column rank, requiring M >_ N .  When A is not full rank, then signals can only be separated as classes [a ] .  In this paper, we only consider systems in which the sources are fully separable. Our objective is to find a signal extracting matrix such that B ~ z ( ~ )  = B ~ A z ( ~ )  4- IS'S(.) = E Z ( ~ )  + B T G ( n )  (3 )  (4) where E = B ~ A  is a permutation matrix which only has one nonzero ele- ment in each row and column. We assume that the source signals ~ ( n )  = [ s l ( n )  s2(n)  . . . s,v(n) I' are uncorrelated of one another but are not white. Hence, and R , ( k )  fi E{s'(n)s'(n - k I T }  # 0 ,  for some k # 0. (6) 2485 0-7803-4428-6/98 $10.00 0 1998 IEEE 3. SIGNAL SEPARABILITY BASED ON Let r,(O) = E{lsl(n)l'). When all signals are non-zero, then denote SECOND ORDER STATISTICS A(k) = R , ( k )  =diag(r~(lc),rz(k), . . . ,  rnr(k)) (7) is full rank. Without loss of generality, we can assume that Given white noise with known correlation matrix A,, R,(O) = I .  we have R,(O) = AR,(0)AT + A ,  = AAT + A,. (8)  Thus, we can use a whitening matrix W such that I = w(R,(o) -A , ) w ~  = W A A ~ W * ,  (9) which implies that U = WA is unitary. We note that the assumption of known noise correlation matrix is typically impractical. In the special case when the noise correlation matrix is of the form a21 and A4 > N, then 0' can be estimated as the smallest eigen-value of Rz(0).  Denote the whitened data vector as 2(n) = WZ(n), with auto-correlation R, (k )  = E{t'(n)z'(n - k ) T } .  Then we have R,(k)  = WAR,(k)ATVVT = U R , ( k ) U T .  (10) Hence columns of U are eigen-vectors of R, (k ) .  If R, ( k )  has identical eigen-values, then their eigen- vectors are not unique. Hence we cannot always find the eigen-vector of R, (k )  as solutions. However, we can sep- arate groups of signals by exploiting eigen-vectors corre- sponding to different eigen-values. In a total blind situa- tion, let R , ( k l )  to have only L distinct eigen-values, then we can have L subspaces that are orthogonal and hence L subclasses of signals can be extracted by U?Z(n),  2 = 1 , 2 , .  . . , L.  (11) The dimension of U, is equal to the multiplicity of the corresponding eigen-value. Once we extract a new signal vector with lower dimen- sion its components can be extracted based on R , ( k z )  using identical steps. To ensure that this procedure can eventu- ally extract all the sources, it is necessary that the sources have different power spectral densities. In [6],  an algorithm which is exponentially convergent is proposed. The algorithm discussed above assumes that the cor- relation matrix of the noise is either known or can be es- timated when n/r > N. However, when the noise corre- lation matrix is either unknown or cannot be estimated, the methods will not apply. In view of this difficulty, we present in the next section an alternative algorithm which is insensitive to additive white noise. &(n)  = U,TZ(n) ,  (12) 4. A NEW MATRIX PENCIL SEPARATOR We now present a method that does not rely on the in- formation in R,(O) which is corrupted by white noise. In- stead, we shall use information contained in R,(IC) = A A ( ~ ) A ~ ,  IC # o that is insensitive to additive white noises. eigen-value problem. If we solve for The algorithm is based on the following generalized effectively the equation can be rewritten as (14) A [ A ( k l )  - XA(kz)AT] v '=  0. One trivial solution to the eigen-vector problem is AT& = 0. (15)  Note that the trivial solution will exist if M > N. Since CT.'(n) = GTAZ(n) + G:G(n) = r$G(n), (16) 50 annihilates all signals in .'(n). In fact, if the noise cor- relation matrix is a21, the trivial solution & can be used to estimate 2 since i$Rz(O)z70 = a'. (17) Since A has full column rank, any other non-trivial so- lution requires eigen-value to satisfy whose eigen-vector must satisfy ATV; = ale,. (19) Therefore, if the ratio of % is unique, then v', is simply the 2-th signal extractor to yield CT.'(n) = CTAZ(n) + 2?,TG(n) = ~ , s , ( n )  + G?G(n). ( 2 0 )  In general, if the ratio of is not unique and then let V, consists of all eigen-vectors for eigen-value 7 v, = [GI Gt2  . . . v',, 1. ( 2 2 )  A ~ V ,  = [e , ,  e , ,  . . . e , ,  ]U,, ( 2 3 )  ?,(n) = V:Z(n) = VTAs'(n) + VTG(n)  (24) = U; [Sli(n) sz2(n) " '  szm(n)] '  + VyT.'(n) We have in which U, is an z, x i, unitary matrix. Hence V, can be used to extract the set of signals {s,, i E I} A Signals within the set I can be extracted from Z7 by applying the same principle on its auto-correlation matri- ces for a pair of different delays, k~ and kg . Alternatively, since U, is unitary, its estimate can be obtained directly as the eigen-vectors of R,(k).  Here we note that the trivial solutions of (15) are not separable from the other general eigen vectors of ( R x ( k l ) ,  R s ( k 2 ) ) .  However, this is not a problem. The ex- tracted signals according to the trivial solutions are pure noises result in the smallest output power while signal ex- tractors will generate stronger outputs. Thus we can al- ways discard the trivial solution as the n/r - N vectors that result in the smallest product v'TR,(O)v'o. R7(k) e VTR,(k)V, = UTA(k )U,  (55) 2486 5 .  SIMULATION Before numerical simulations are presented. It is helpful l,o analyze the best possible results one can obtain in signal separation when A is known a priori. Defining the pseudo- inverse of A as A’,  the signal estimate should be $n)  = A t Z ( n )  = g(n) + A’G(n). This represents the best possible result when noise is wn- known. Experimental results will be compared with the ideal situation. When A is unknown, blind separation performance will naturally be inferior due to two reasons: e The data length may be too short to accurately esl,i- e The underlying sources may not be fully uncorrelated with one another. This violation of our basic assump- tion will also degrade the performance. mate the correlation matrices R,(k).  In the simulation, we consider a system with four sources and five outputs. Two sets of signals are tested. The first set of signals are four random color Gaussian sig- nals. Each signal is generated by filtering a white Gaussian noises using a 20-tap FIR filter with impulse response where cy = 1,0.4,0.3,0.2 respectively. The second set of signals are four speech signals sampled at  5500%. The mixing matrix used in both examples is 0.6107 0.4959 0.2661 0.2504 0.2380 0.2887 -0.7120 0.4914 0.3397 -0.7494 -0.1157 0.2097 0.3558 0.2644 -0.4216 -0.6640 -0.5731 -0.1983 -0.4807 0.4593 A =  [ The columns of A are normalized so that the total power of each sources in the mixtures to be equal. Simulations were conducted for 3 different signal lengths of 1000, 4000, and 16000, with individual input signal to noise ratio (SNR) at  -10, 0, 10, 20 30, and 40dB. For each situation, we run 100 Monte Carlo simulations to get the statistical performance. The individual input SNR is now identical for all sources since we normalized columns of A. This allows a fair coin- parison. To measure the statistical performance of our method, we use the output signal to interference and noise ratio (SINR) as the performance index. The delay p<iir used in the matrix pencil method is ( k l  = I, IC2 = 3). The output SINR according to the individual input SNR are given in Figure (1 and Figure (2) for the two examples of signal separation, respectively. The output SINR is averaged over 100 Monte Carlo simulations. Let L be the length of data samples. In each figure, circle ‘0’ denotes cases for L = 1000, x-mark ‘x’ for L=4000, and plus ‘+’ for L = 16000. Finally to compare with the upper bound of output SINR, solid lines are given for ideal cases in which the mixing matrix A is known It can be seen that the matrix pencil method works well for real world speech signals. especially Comparing results from different data lengths, we find that the performance at L =lo00 is very close to what is achieved at  length L ~ 1 6 0 0 0 .  This result demonstrates that the proposed method converges very fast since the estimation of second order statistics does not require many data samples. For the specific examples, the output SINR saturates at about 20dB. This is because in high SNR case, the perfor- mance is dominated by the level of cross interference which is determined by the cross correlation between sources and the mixing matrix. Clearly for the same mixing matrix A ,  different signal sources with different cross correlation also result in different output SINR. 6 .  CO’NCLUSION In this paper we present a new second order statistical method for blind source separation. I t  is based on the general eigen decomposiltion of the matrix pencil formed between output auto-correlation matrices a t  different de- lays. The method has several attractive features compared with higher order statistical methods. First, it  relies only on second order statistics of the received signals and hence requires fewer data samples to converges. Second , it can separate Gaussian sourc’es. Compared with other known methods based on second order statistics, it does not re- quire zero additive noise or that the noise correlation ma- trix is known, and it is unbiased. Numerical simulations show that the method .works well for real world speech signals. A C K N O W L E D G M E N T  The work of Chang and Chan was supported by in part by grants from Research Grants Council of Hong Kong, grant numbers HKU55:3/96M and HIW7036/97E. The work of Ding was supported in part by the US Army Re- search Office and by the visiting appointment at HKUST. The work of Yau was supported in part by grants from Research Grants Council of Hong Kong, grant number HIiUST776/96E. R E F E R E N C E S  [I] P. Comon, “Independent component analysis, a new concept?” Signal Processing, vol. 36, pp. 287 - 314, Apr. 1994. [a] X. Cao and R. Liu, “General approach to blind source separation”, IEEE Xruns. Signal Processing, vol. 44, pp. 562 - 571, hhr .  1.996. [3] J.F. Cardoso, “Source separation using higher order moments”, Proc. In!:. Con$ o n  Acoustics, Speech and Signal Processing, pp. 2109 - 2112, Glasgow, UK, 19S9. [4] L. Tong, R. Liu, V.C. Soon, and Y. Huang, “Indetermi- nacy and identifiability of blind identification”, IEEE Trans. Circuits and Systems, vol. 38, pp. 499 - 509, May 1991. [5] A. Belouchrani, I i .  Abed-Meraim, J.F. Cardoso, and E. Moulines, “A blind !source separation technique using second-order statistiNcs”, I E E E  Trans. Signal Process- ing, vol. 45, pp. 43-1-4114, Feb. 1997. [6] C. Chang, S.F. Yau, 1’. I<wok, F.K. Lam, and F.H.Y. Chan, “Sequential approach to blind source separation using second order :statistics”, Proc. of Internationul Conference o n  Information, Communication and Sig- nul Processing, pp. 1608-1612, Sep. 1997. (a) The 1st source _. 40 30 Ib) The 2nd source . . . . . . . .  .:. . . . . . . . . . . . . . . .  : . . . . . . . .  .:. . . . . . . .  :. . . . . .  20 10 / .:. . . . . . . . . . . .  . . . . . . . . . . .  . . . . .  ...:. . . . . . A .  . . . . . . . . . . . . . . .  o . . .  . . . .  8 . . . . . . . . . . . . .  . . ;  . . . .  . . I  . . . . . .  R 0 -1 -10 0 10 20 30 40 Input SNR (dB) ." -10 0 10 20 30 40 Input SNR (dB) (c) The 3rd source (d) The 4th source -10 0 10 20 30 40 -10 0 10 20 30 40 Input SNR (dB) Input SNR (dB) Figure 1. Performance for randomly generated sources fa) The 1st source (b) The 2nd source -10 0 10 20 30 40 Input SNR (dB) (c) The 3rd source 1 . -  -10 0 IO 20 30 40 Input SNR (dB) 40 . h m 9 3 0 -  lx .., -10 0 10 20 30 40 Input SNR (dB) (d) The 4th source -1 0 -10 0 10 20 30 40 Input SNR (dB) Figure 2. Performance for real world speech sources 2488 

A matrix-pencil approach to blind separation of non-white sourcesin white noise

http://hub.hku.hk/bitstream/10722/46052/1/34819.pdf

A matrix-pencil approach to blind separation of non-white sourcesin white noise

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub

HKU Scholars Hub