In this paper, we consider the problem of remote vector Gaussian source
coding for a wireless acoustic sensor network. Each node receives messages from
multiple nodes in the network and decodes these messages using its own
measurement of the sound field as side information. The node's measurement and
the estimates of the source resulting from decoding the received messages are
then jointly encoded and transmitted to a neighboring node in the network. We
show that for this distributed source coding scenario, one can encode a
so-called conditional sufficient statistic of the sources instead of jointly
encoding multiple sources. We focus on the case where node measurements are in
form of noisy linearly mixed combinations of the sources and the acoustic
channel mixing matrices are invertible. For this problem, we derive the
rate-distortion function for vector Gaussian sources and under covariance
distortion constraints.Comment: 10 pages, to be presented at the IEEE DCC'1

Bech, Soren

Jensen, Soren Holdt

Naylor, Patrick

Ostergaard, Jan

Zahedi, Adel

English

arXiv

Adel Zahedi

Jan Ostergaard

Soren Holdt Jensen

Patrick Naylor

Soren Bech

Crossref

Distributed Remote Vector Gaussian Source Coding for Wireless Acoustic Sensor Networks

In this paper, we consider the problem of remote vector Gaussian source coding for a wireless acoustic sensor network. Each node receives messages from multiple nodes in the network and decodes these messages using its own measurement of the sound field as side information. The node's measurement and the estimates of the source resulting from decoding the received messages are then jointly encoded and transmitted to a neighbouring node in the network. We show that for this distributed source coding scenario, one can encode a so-called conditional sufficient statistic of the sources instead of jointly encoding multiple sources. We focus on the case where node measurements are in form of noisy linearly mixed combinations of the sources and the acoustic channel mixing matrices are invertible. For this problem, we derive the rate-distortion function for vector Gaussian sources and under covariance distortion constraints.In this paper, we consider the problem of remote vector Gaussian source coding for a wireless acoustic sensor network. Each node receives messages from multiple nodes in the network and decodes these messages using its own measurement of the sound field as side information. The node’s measurement and the estimates of the source resulting from decoding the received messages are then jointly encoded and transmitted to a neighboring node in the network. We show that for this distributed source coding scenario, one can encode a so-called conditional sufficient statistic of the sources instead of jointly encoding multiple sources. We focus on the case where node measurements are in form of noisy linearly mixed combinations of the sources and the acoustic channel mixing matrices are invertible. For this problem, we derive the rate-distortion function for vector Gaussian sources and under covariance distortion constraints

Østergaard, Jan

Jensen, Søren Holdt

Bech, Søren

    Aalborg UniversitetDistributed Remote Vector Gaussian Source Coding for Wireless Acoustic SensorNetworksZahedi, Adel; Østergaard, Jan; Jensen, Søren Holdt; Naylor, Patrick; Bech, SørenPublished in:Data Compression Conference (DCC), 2014DOI (link to publication from Publisher):10.1109/DCC.2014.27Publication date:2014Document VersionEarly version, also known as pre-printLink to publication from Aalborg UniversityCitation for published version (APA):Zahedi, A., Østergaard, J., Jensen, S. H., Naylor, P., & Bech, S. (2014). Distributed Remote Vector GaussianSource Coding for Wireless Acoustic Sensor Networks. In Data Compression Conference (DCC), 2014 (pp. 263-272). IEEE Press.  (Data Compression Conference. Proceedings). DOI: 10.1109/DCC.2014.27General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.            ? Users may download and print one copy of any publication from the public portal for the purpose of private study or research.            ? You may not further distribute the material or use it for any profit-making activity or commercial gain            ? You may freely distribute the URL identifying the publication in the public portal ?Take down policyIf you believe that this document breaches copyright please contact us at vbn@aub.aau.dk providing details, and we will remove access tothe work immediately and investigate your claim.Downloaded from vbn.aau.dk on: April 29, 2017Distributed Remote Vector Gaussian SourceCoding for Wireless Acoustic Sensor NetworksAdel Zahedi∗, Jan Østergaard∗, Søren Holdt Jensen∗, Patrick Naylor†,and Søren Bech∗‡∗Department of Electronic SystemsAalborg University, 9220 Aalborg, DenmarkEmail: {adz, jo, shj, sbe}@es.aau.dk†Electrical and Electronic Engineering DepartmentLondon Imperial College, London SW7 2AZ, UKEmail: p.naylor@imperial.ac.uk‡Bang & Olufsen7600 Struer, DenmarkAbstractIn this paper, we consider the problem of remote vector Gaussian source coding for awireless acoustic sensor network. Each node receives messages from multiple nodes in thenetwork and decodes these messages using its own measurement of the sound field as sideinformation. The node’s measurement and the estimates of the source resulting from decodingthe received messages are then jointly encoded and transmitted to a neighboring node in thenetwork. We show that for this distributed source coding scenario, one can encode a so-calledconditional sufficient statistic of the sources instead of jointly encoding multiple sources. Wefocus on the case where node measurements are in form of noisy linearly mixed combinationsof the sources and the acoustic channel mixing matrices are invertible. For this problem, wederive the rate-distortion function for vector Gaussian sources and under covariance distortionconstraints.The research leading to these results has received funding from the European Union’s Seventh Framework Programme(FP7/2007-2013) under grant agreement n◦ ITN-GA-2012-316969.I. INTRODUCTIONA Wireless Acoustic Sensor Network (WASN) is a set of wireless microphones equippedwith some communication, signal processing, and possibly memory units, which arerandomly scattered in the environment. The communication unit allows a sensor node tocommunicate with a base station and also with other nodes, and the signal processing unitenables a node to perform local processing. The random structure of the network removesthe array-size limitations imposed by classical microphone arrays, thereby providingbetter performance in sense of high-SNR spatio-temporal sampling of the sound fieldand spatial diversity. The interested reader is referred to [1] for a review of WASNs.Due to strict power and bandwidth constraints on wireless microphones, it mightnot be possible for nodes which are far from the base station to directly deliver theirmessages. A multi-hop scenario where the message goes along adjacent nodes untilreaching the base-station is a commonplace alternative. We assume that a node is supposedto receive messages from multiple neighboring nodes and then combine them with itsown measurement of the sound field into a new message to be forwarded through thenetwork. This is illustrated in Fig.1.In [2], this problem was solved for scalar Gaussian sources in a non-distributedsetting i.e., without making use of the availability of the nodes’ measurements as sideinformation. It was shown that coding at intermediate nodes can result in significant gainsin terms of sum-rate or distortion. In this paper, we consider vector Gaussian sourcesand also take into account the fact that messages received by a node are correlated withthe node’s measurement of the sound field. Thus, we consider a distributed scenario,and make use of the destination node’s measurement as side information to decrease therequired rate for transmission to each node [3], [4], [5].We derive the rate-distortion (RD) function for an arbitrary node in the network witha distortion constraint defined in form of a covariance matrix, cf. [6], [7]. In Section II,we introduce the notation and formulate the problem, which turns out to involve jointcoding of multiple sources. In Section III, we derive a conditional sufficient statisticfor multiple Gaussian sources and show that for the above-mentioned distributed sourcecoding (DSC) problem, one can encode a conditional sufficient statistic instead of jointencoding of multiple sources. The RD function for the resulting problem will be derivedin Section IV. The paper is concluded in Section V.II. NOTATION AND PROBLEM FORMULATIONWe denote by I(·; ·), h(·), and E[·] the information theoretic operations of mutualinformation, differential entropy, and expectation, respectively. Probability density func-tions are denoted by f(·) and covariance and cross-covariance matrices are denoted byΣ. We assume that all covariance matrices are of full rank. We denote Markov chains bytwo-headed arrows; e.g. u↔ v↔ w. We assume that the source generates independentGaussian vectors of length n denoted by x ∈ Rn. While the vectors are independent, thecomponents of each vector are correlated. Node j makes a noisy measurement yj of thesource given by:yj = Ajx + nj, (1)Figure 1: A schematic graph of the WASNwhere nj is the additive Gaussian noise at node j and the matrix Aj models the mixingeffect of the acoustic channel. The noise is assumed to be independent of x. Node jalso receives messages from nodes i, l, ... denoted by ui,ul, ..., from which it can makeestimations xˆi, xˆl, ... of the source by decoding the messages with yj as the decoder sideinformation. The problem is to find the minimum rate Rj to jointly encode yj, xˆi, xˆl, ...into a message uj to be sent to node k, while satisfying the given distortion constraintDj and considering yk as side information. This is illustrated in Figs.1 and 2.Note that one could further decrease the network sum-rate by taking into account thecorrelation between the messages which are sent to a common node. However, we leaveout this possibility in this paper.Throughout this work, we assume that the acoustic mixing matrices are fixed andknown. We also assume that joint statistics of the source and the noise variances areavailable. Finally, although the model in (1) is appropriate for acoustic networks, we donot consider real acoustic signals in this work. Instead, we consider Gaussian sources forsimplicity of mathematical analysis. The case of real audio measurements is the focus ofour future work.III. CONDITIONAL SUFFICIENT STATISTICSAssume that y is a random vector or a collection of random vectors with probabilitydensity function f(y).Definition 3.1: A sufficient statistic for the estimation of x from y is a function T (y)of y for which f(y|T (y),x) is not a function of x. In other words, y↔ T (y)↔ x.Theorem 3.1 (Neyman-Fisher Factorization Theorem [8]): T (y) is a sufficient statis-tic of y for estimating x if and only if f(y|x) can be factorized as:f (y|x) = p (y) q (T (y) ,x) , (2)where p (depending on y only) and q (depending on x and T (y) but not on y directly)are nonnegative functions.Figure 2: Node j of the networkThe Factorization Theorem enables us to find a sufficient statistic for e.g. randomvectors in Gaussian noise as shown in the following lemma1.Lemma 3.1: If yj; j = 1, 2, ..., N are measurements of a random vector x in mutuallyindependent Gaussian noises as in (1), then T (y1, ...,yN) is a sufficient statistic of yjfor estimating x, whereT (y1, ...,yN) =N∑j=1ATj Σ−1j yj, (3)and Σj is the covariance matrix of the noise nj .Proof: We prove the lemma for N = 2. The proof for the general case is similar.Since n1 and n2 are independent, the joint conditional density function of y1 and y2can be written as:f (y1,y2|x) = α exp(−12ϕ), (4)where α is independent of x, y1 and y2, and ϕ is defined as:ϕ = (y1 − x)TΣ−11 (y1 − x) + (y2 − x)TΣ−12 (y2 − x) . (5)Expanding ϕ, rearranging the terms, and substituting in (4), leads tof (y1,y2|x) = α exp(−yT1 Σ−11 y1 + yT2 Σ−12 y22)1To the best of our knowledge, no proof for this lemma appears in the literature. We provide a proof here forcompleteness.× exp(−xT(Σ−11 + Σ−12)x2)exp(T Tx + xTT2), (6)which shows that T is a sufficient statistic according to the Factorization Theorem.To make the notion of sufficient statistics applicable to our DSC problem, we need tointroduce a conditional version:Definition 3.2: A conditional sufficient statistic for the estimation of x from y givenz is a function T (y) of y for which f(y|T(y),x, z) is not a function of x.Lemma 3.2: If y ↔ x ↔ z, then any sufficient statistic T (y) of y for estimating xis also a conditional sufficient statistic of y for estimating x given z.Proof:f (y|T (y) ,x, z) = f (y|x, z)f (T (y) |x, z)=f (y|x)f (T (y) |x)= f (y|T (y) ,x)= f (y|T (y)) (7)which is independent of x.Lemma 3.3: Given side information z at the decoder, there is no loss in terms of rateand distortion by encoding a conditional sufficient statistic of y1, ...,yN given z insteadof joint encoding of these sources.Proof: The proof follows along the lines of the proof for the unconditional casepresented in [9], and is therefore omitted.Combining the results of this section, we have the following theorem for the problemformulated in Section II:Theorem 3.2: Given side information yk at the decoder, the RD function for theproblem of joint encoding of multiple sources yj, xˆi, xˆl, ... coincides with the RD functionfor the DSC problem of encoding the single source:Tj = ATj Σ−1j yj + D−1i xˆi + D−1l xˆl + ... (8)where Di is the distortion matrix for node i.Proof: Using the backward channel model we have x = xˆi+qi where the covariancematrix of qi is Di. This means that xˆi can be written as:xˆi = Hix + ηi, (9)whereHi = (Σx −Di)Σ−1x , (10)Σηi = (Σx −Di)Σ−1x Di. (11)Substituting (10) and (11) in (9) and using (3) yields (8). Since Tj is a sufficient statisticof yj, xˆi, xˆl, ... and yj, xˆi, xˆl, ... ↔ x ↔ yk, it follows from Lemma 3.2 that it isalso a conditional sufficient statistic given yk. From Lemma 3.3, one can then replaceyj, xˆi, xˆl, ... by Tj in (8) and get the same RD function.At this point, we have shown that the above problem with multiple sources canbe converted into a single source DSC problem for Gaussian sources with covariancedistortion constraint. This problem is illustrated in Fig.3. For the case of mean-squarederror distortion constraint, the RD function was found in [10], while the case of covariancedistortion was not treated in that work. In the next section, we derive the RD functionfor the covariance distortion constraint under some mild technical assumptions.IV. RATE-DISTORTION FUNCTIONFor the sake of simplicity of derivations, we write x and Tj in terms of their linearestimations based on the known Gaussian vectors in Fig.3. In particular, we have thatTj = Ayk + Buj + υ1, (12)x = CTj + Gyk + υ2 = (G + CA)yk + CBuj + Cυ1 + υ2, (13)Tj = Γyk + υ3, (14)where υ1, υ2, υ3 are estimation errors with covariance matrices ΣTj |yk,uj ,Σx|Tj ,yk ,ΣTj |yk ,respectively, and A,B,C,G and Γ depend only on the covariance and cross-covariancematrices of x, Tj,yk and uj . (See Appendix for the mathematical statement for C as anexample.) We will show that if the mixing matrices Aj in (1) are invertible2, then forΣx|Tj ,yk ≺ Dj  Σx|yk the RD function for node j is given by:Rj (Dj) =12log(∣∣Σx|yk −Σx|Tj ,yk∣∣∣∣Dj −Σx|Tj ,yk∣∣). (15)2The mixing matrices are actually non-square and thus non-invertible. In this work we consider a squareapproximation. The general non-square case will be considered in our future work.Figure 3: The simplified DSC problemA. Lower BoundFirst we need the following lemma:Lemma 4.1: If the mixing matrices Aj, j = 1, 2, ..., are invertible, the matrix C in(13) is also invertible.Proof: See the appendix.From the first equality in (13) we have:CΣTj |ykCT = Σx|yk −Σx|Tj ,yk . (16)From the second equaltiy in (13) we can write:x− xˆk = Cυ1 + υ2. (17)Assume now that the sequence of independent vectors xi generated by the source isencoded in block vectors X each containing N vectors. From the distortion constraintand (17) we can write:Dj  1NN∑i=1E[(xi − xˆik) (xi − xˆik)T]= CΣυ1CT + Συ2 (18)or equivalently:CΣTj |uj ,ykCT  Dj −Σx|Tj ,yk . (19)Denoting the blocks of N vectors T ij ,yik,uij; i = 1, ..., N by Tj,Yk,Uj , respectively,we have:NRj (Dj) ≥ I (Tj;Uj|Yk)= h (Tj|Yk)− h (Tj|Uj,Yk)=N∑i=1h(T ij |yik)− h (T ij |Ti−1j ,uij,yik)≥N∑i=1h(T ij |yik)− h (T ij |uij,yik) (20)=N2log( ∣∣ΣTj |yk∣∣∣∣ΣTj |uj ,yk∣∣)=N2log( ∣∣CΣTj |ykCT ∣∣∣∣CΣTj |uj ,ykCT ∣∣)(21)≥ N2log(∣∣Σx|yk −Σx|Tj ,yk∣∣∣∣Dj −Σx|Tj ,yk∣∣), (22)where the block vector Ti−1j contains i − 1 vectors T 1j , ..., T i−1j , (20) is because con-ditioning reduces the entropy, (21) is because C is invertible, and (22) is the result ofapplying (16) and (19) to (21).B. Upper BoundThe lower bound derived in the previous part can be used as a guideline for the bestpossible performance for any coding scheme. The question is then, given the source Tj ,how to encode it into a discrete source uj , so that for the given distortion constraint, therequired rate for transmission of uj achieves the lower bound?Let us assume that Tj is quantized to uj , in a way that satisfies the distortion constraint.From the results of DSC, due to the availability of the side information yk at the decoder,it is possible to noiselessly encode uj in blocks of sufficiently large length N with a ratearbitrarily close to H(uj|yk) = I (Tj;uj|yk) [3], [4], [5]. The remaining task is to designa scheme for which I (Tj;uj|yk) achieves (15). This is possible by using the followingscheme:uj = UCTj + νj, (23)where we have denoted the eigenvalue decomposition of(Σx|yk −Σx|Tj ,yk)by UTΛU,and the covariance Σνj of the coding noise νj is defined as3:Σνj = U(Σx|yk −Σx|Tj ,yk)(Σx|yk −Dj)−1(Dj −Σx|Tj ,yk)UT, (24)To verify this, one can write I (Tj;uj|yk) as h (uj|yk)−h (uj|Tj,yk), substitute (23)–(24),and utilize (14) and (16).It is worth noting that the RD function (15) generalizes the RD function of [11], whichtreated the scalar case.3Note that it is a symmetric positive definite matrix.V. CONCLUSIONS, DISCUSSIONS, AND FUTURE WORKWe showed that the rate-distortion function for a distributed source coding problemwith multiple sources at the encoder is identical to the rate-distortion function for thedistributed encoding of a so-called conditional sufficient statistic of the sources. Wederived a conditional sufficient statistic for the case that additive noises are Gaussianand mutually independent, and calculated the rate-distortion function in case that thesources are vector Gaussian and the distortion constraint is defined as a covariance matrix.Since vector sources were considered in order to take the memory into account, and thecovariance constraint on the distortion is a more flexible fidelity criterion compared tomean-squared error, these results can be applied to the problem of source coding foraudio signals in presence of reverberation, which will be the scope of our future work.APPENDIXAssume that the Gaussian vector x is estimated using two other Gaussian vectors yand z as:x = Cy + Gz + n, (25)where the covariance matrix of the estimation error n is Σn = Σx|yz, and y and z arerelated to x via y = Ax + n1 and z = Bx + n2, and A and B are invertible. Thereforewe have:Σxy = ΣxAT , (26)Σy = AΣxAT + Σn1 , (27)Σyz = AΣxBT , (28)Σxz = ΣxBT . (29)From results in linear estimation theory, we obtain the following expression for C [12]:CΣy = Σxy +(ΣxyΣ−1y Σyz −Σxz)∆−1ΣTyz, (30)∆ = Σz −ΣTyzΣ−1y Σyz. (31)The goal is to prove that C or equivalently CΣy is invertible. Substituting (26)–(29) in(30) yields:CΣy = ΣxAT +(ΣxATΣ−1y AΣxBT −ΣxBT)∆−1ΣTyz= ΣxAT + Σx(ATΣ−1y AΣx − I)BT∆−1ΣTyz= ΣxAT + ΣxAT(Σ−1y −A−TΣ−1x A−1)AΣxBT∆−1ΣTyz= ΣxAT + ΣxAT[(AΣxAT + Σn1)−1 − (AΣxAT )−1]Σyz∆−1ΣTyz= ΣxAT(I + H−1Σyz∆−1ΣTyz), (32)where H is defined as:H−1 =(AΣxAT + Σn1)−1 − (AΣxAT )−1. (33)From (31) and (26)–(29) we have:∆ = BΣxBT + Σn2 −BΣxAT(AΣxAT + Σn1)−1AΣxBT= BΣx[I−AT (AΣxAT + Σn1)−1AΣx]BT + Σn2= BΣxAT[A−TΣ−1x A−1 − (AΣxAT + Σn1)−1]AΣxBT + Σn2= ΣTyz(−H−1)Σyz + Σn2 . (34)Substituting (34) in (32) and defining P as:P−1 =(ΣTyzH−1Σyz)−1 − (ΣTyzH−1Σyz −Σn2)−1, (35)we have:CΣy = ΣxATH−1ΣyzP−1ΣTyz, (36)which is clearly invertible.REFERENCES[1] A. Bertrand, Applications and trends in wireless acoustic sensor networks: a signal processing perspective, inProc. IEEE Symposium on Communications and Vehicular Technology (SCVT), Nov. 2011.[2] J. Østergaard and M. S. Derpich, Sequential Remote Source Coding in Wireless Acoustic Sensor Networks,European Signal Processing Conference (Eusipco), pp. 1269-1273, Aug. 2012.[3] D. Slepian and J.Wolf, Noiseless coding of correlated information sources, IEEE Transactions on InformationTheory, vol. 19, no. 4, pp.4711-480, Jul. 1973.[4] A. Wyner and J. Ziv, The rate-distortion function for source coding with side information at the decoder, IEEETransactions on Information Theory, vol. 22, no. 1, pp.1-10, Jan. 1976.[5] A. Wyner, The rate-distortion function for source coding with side information at the decoder- II: General sources,Information and Control, vol. 38, no. 1, pp.60-80, Jul. 1978.[6] M. S. Rahman and A. Wagner, Rate region of the Gaussian scalar-help-vector source-coding problem, IEEETransactions on Information Theory, vol. 58, no. 1, pp.172-188, Jan. 2012.[7] G. Zhang, On the rate region of the vector Gaussian one-helper distributed source-coding problem, in Proc. DataCompression Conference, 2011.[8] S. Kay, Fundamentals of Statistical Signal Processing, vol. I: Estimation Theory, Prentice Hall, 1st edition, 1993.[9] K. Eswaran and M. Gastpar, Rate loss in the CEO problem, in 39th Annual Conference on Information Sciencesand Systems, Baltimore, MD, 2005.[10] C. Tian and J. Chen, Remote vector Gaussian source coding with decoder side information under mutualinformation and distortion constraints, IEEE Transactions on Information Theory, vol. 55, no. 10, pp.4676-4680,Oct. 2009.[11] S. C. Draper and G. W. Wornell, Side information aware coding strategies for estimation under communicationconstraints, IEEE Journal on Selected Areas in Communications, vol. 22, no. 6, pp. 1-11, Aug. 2004.[12] T. Kailath, A. Sayed, and B. Hassibi, Linear Estimation, Prentice-Hall, 2000.

http://vbn.aau.dk/files/209616146/Coding_for_WASN_DCC2014.pdf

Distributed Remote Vector Gaussian Source Coding for Wireless Acoustic
  Sensor Networks

Distributed Remote Vector Gaussian Source Coding for Wireless Acoustic Sensor Networks

Abstract

Similar works

Full text

Available Versions

Crossref

VBN