Publication in the conference proceedings of EUSIPCO, Florence, Italy, 200

Dabeer, Onkar

Karnik, Aditya

English

ZENODO

CONSISTENT SIGNAL PARAMETER ESTIMATION WITH 1-BIT DITHEREDSAMPLINGOnkar Dabeer, Aditya KarnikSchool of Technology and Computer Science Dept. of Electrical and Computer Engg.Tata Institute of Fundamental Research University of WaterlooColaba, Mumbai 400 005 India Waterloo, N2L 3G1, Canadaonkar@tcs.tifr.res.in karnik@ece.waterloo.caABSTRACTWe consider the problem of estimating a parameter θ of asignal s(x;θ) corrupted by noise when only 1-bit precisionsamples are allowed. We propose and analyze a new estima-tor based on dithered 1-bit samples. Our estimate is consis-tent and satisfies an asymptotic CLT for a wide class of ditherdistributions. In particular, uniformly distributed dither leadsto only a logarithmic rate loss compared to the case of fullprecision samples.1. INTRODUCTIONThis paper is concerned with the problem of estimating a pa-rameter of a signal corrupted by noise when only its 1-bitprecision samples are available. This problem is particularlymotivated by the possibilities that have opened up with thedevelopment of miniature sensing devices which can com-pute and communicate autonomously. Networks of such“smart” sensors could be used for detecting and estimatingcharacteristics of spatio-temporal processes in several situ-ations ([4]). When physical models for such processes areknown, these problems can often be cast as those of estima-tion of parameters of a signal; for example, estimating thevelocity of a wave propagating through some medium. Theconstraint posed by this exciting sensing technology, how-ever, is that being miniature and low cost, a typical sensorwill be a low precision device; hence, any estimation schemein this context can employ only low precision, noisy mea-surements available from each sensor.Formally, the problem is to estimate a deterministic butunknown parameter θ0 ∈ R of a real-valued signal s(x;θ0)corrupted with additive noise. In this paper, we consideronly one dimensional space and no time variation. We con-sider bounded observation space: x ∈ [0,1]. Results simi-lar to ours can be obtained for the general case with mi-nor notational changes. To make the problem well-posed,we assume throughout that this problem has a unique so-lution when s(x;θ0) is observed without any noise, that is,s(x;θ) 6= s(x;θ ′) whenever θ 6= θ ′.When full precision samples with i.i.d. Gaussian noiseare available at N locations, then the maximum-likelihood(ML) estimate of θ0 is well analyzed. Let the observationsbe denoted byZ(n) = s( nN;θ0)+σtWt(n)where {Wt(n),n= 1, ...,N} is the i.i.d. thermal/ambient noisein the measurements with common distribution N (0,1).The maximum likelihood (ML) estimator of θ0 seeks a least-squares fit to the observations and the ML estimate is a solu-tion of the log-likelihood equation (see [7, pp. 187])N∑n=1s′( nN;θ)[Z(n)− s( nN;θ)]= 0. (1)Under mild regularity conditions, the ML estimate is consis-tent (in probability as N → ∞), satisfies a central limit theo-rem (CLT), and the variance in the CLT attains the Cramer-Rao lower bound.In contrast when only 1-bit precision samples of obser-vations ,i.e., sign(Z(n)), are available, the ML equation iscomplicated and little is known about the performance ofthe ML estimate for the general case. For example, whens(x;θ) = θ , then in the full precision case the ML estimateis the sample mean, but no closed form solution is known forthe ML estimate in the 1-bit sampling case. In this paper,we employ dithered 1-bit sampling and propose an estimatethat requires solving an equation of the form (1). In ditheredsampling, a random noise is added to Z(n) before quantizingit. Thus, our method leverages the idea of obtaining a lowprecision measurement at each of the possibly large numberof sensors and combining them to form an estimate of the pa-rameter1. Moreover, our estimator requires no more compu-tation than the ML estimate in the full precision case, and forthe above example of s(x;θ) = θ , our estimate just involvesthe averaging of dithered samples. Our estimate is consistentand asymptotically normal for a wide class of dither distri-butions. In particular, uniform dithering leads to only a log-arithmic rate loss compared to the full precision case. Dueto space constraint, we do not present the full mathematicalderivations in this paper (these can be found in [3]). But wediscuss the intuition behind our results with examples andsimulations.1.1 Related Prior WorkWhile a theory of estimation using full precision sampling isnow well-established (see for example, [7, Section IV.E.2]),the case of low precision sampling seems to have been con-sidered only for particular cases. For example, the problemof frequency estimation using 1-bit ADC has been studiedin [5], [10]. In [9], the problem of choosing the quantiza-tion threshold for the case of signal amplitude estimation isinvestigated.To the best of our knowledge, the problem of estimat-ing θ0 for a general signal s(x;θ0) with only 1-bit ADC has1We do not discuss strategies for distributed implementation of thisscheme in this paper.14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIPnot been addressed before in the literature. However, a con-sistent estimator using only 1-bit ADC can be derived fromreference [6], which addresses the problem of estimating thesignal s(x;θ0) from the observations {sign(Z(n))}. Our es-timator is an improvement over this estimator: a) it requiresless computation, and b) it has a better rate for the CLT (seeSection 3.1 for a discussion).A key element in our results is dithering. Dithering hasbeen used previously by several researchers in different con-texts; for example, [6] uses it to estimate a smooth signal us-ing only 1-bit ADC, [2] employs a deterministic dither signaland oversampling to recover a band-limited function from fi-nite precision samples. However, in the context of signalparameter estimation, dithering does not seem to have beenexploited before.2. OBSERVATION MODEL AND ASSUMPTIONSThe observations {Y (n), n = 1, ...,N} are given byY (n) = sign(s(Xn;θ0)+σtWt(n)+a(N)Wd(n)) (2)The random variables {Wd(n)} constitute the dither signal,while {Xn} are the sampling locations. We estimate θ0 bysolving TN(θ) = 0, where,TN(θ) =1NN∑n=1s′ (Xn;θ) [cd(N)Y (n)− s(Xn;θ) .] (3)The value of cd(N) depends on the distribution of the ditherand is specified below. Recall that N denotes the number ofspace samples or simply the number of sensors. Below wecollect together most of the assumptions we need for analyz-ing the estimator and we also discuss their implications.Noise: The usual assumption about the noise is:GN) {Wt(n),n ≥ 1} are i.i.d.N (0,1).We assume that the observation noise is dominated by thethermal noise in the sensor circuitry which justifies our i.i.d.noise assumption. (If the noise at different sensors is cor-related, then the situation is best modeled as estimation ofparameter of a stochastic process. This is a tough problemthat we do not address here.)Signal: The signal is assumed to satisfy the following regu-larity constraints.A1) The signal s(x;θ) is thrice differentiable in θ and once inx.A2) The signal |s(x;θ)| ≤ b < ∞ for all x ∈ [0,1] and θ ∈Θ. Similarly, its three derivatives w.r.t. θ and the onederivative w.r.t. x are bounded by a constant that does notdepend on x and θ .A3) The functionJ(θ ;θ0) :=∫ 10s′(x;θ)[s(x;θ0)− s(x;θ)]dxhas a unique zero-crossing at θ = θ0.The assumption A3) ensures that the parameter estimationproblem is well-posed. It basically says that∫ 10[s(x;θ0)− s(x;θ)]2dxhas a unique minimum at θ0, that is, if there is no noise andwe collect full precision data over the entire space [0,1], thenwe can estimate the parameter uniquely. For example, whens(x;θ) = θ p(x), then J(θ ;θ0) = (θ0−θ)∫ 10 p2(x)dx and A3)is seen to be satisfied.Sampling: Our results can be established for deterministic aswell as random spatial sampling. Here we report our resultsonly under:DS) Uniform deterministic sampling: Xn = n/N, 1 ≤ n ≤ N.Other sampling designs are also of importance and can alsobe accommodated. If a non-uniform deterministic samplingis given by Xn = B(n/N), where B : [0,1]→ [0,1], then ourresults under DS) can be applied by redefining the signal tobe s(B(x);θ).Dithering: For the dither signal, we present results under thefollowing two assumptions.UD) The dither signal {Wd(n),1 ≤ n ≤ N} is i.i.d. with uni-form distribution on [−1,1] and it is independent of thethermal noise {Wt(n),1 ≤ n ≤ N}. The dither magnitudeis taken to bea(N) = β (log(N))(1+η)/2 for some β > 0,η > 0The constant cd(N) in (3) is taken to be a(N).NUD) The dither signal {Wd(n),1 ≤ n ≤ N} is i.i.d. with dis-tribution F(t) and it is independent of the thermal noise{Wt(n),1 ≤ n ≤ N}. The distribution F(t) has the fol-lowing expansion for some ε > 0, |t| < ε and someq ∈ {2,3, ...}F(t) =12+b12t +bq2tq +R(t)where b1 6= 0, bq 6= 0, and |R(t)| ≤ constant · |t|q+1 for|t|< ε . The dither magnitude a(N)→∞ and a2(N)/N →0 as N → ∞. The constant cd(N) in (3) is taken to bea(N)/b1.The assumption UD) is of special importance because it leadsto best rates in the CLT for the parameter estimate. In prac-tice, uniformly distributed dithering can be implemented byvarying the threshold of the quantizer by a sawtooth wave.As long as the phases of the sawtooth wave at the differ-ent sampling locations are independent and uniformly dis-tributed, the i.i.d. random dither assumption is justified.3. MAIN RESULTS3.1 Consistency and CLT for the Estimate Under Uni-form DitheringOur first main result is the following.Theorem 1 Suppose the observations are as in (2) and theconditions GN), A1)-A3), DS), UD) are satisfied. Then asN → ∞, there exists a sequence of estimates { ˆθN} such that1. P(TN( ˆθN) = 0)→ 1;2. the estimate ˆθN → θ0 almost surely;3. the following CLT holds√N(logN)(1+η)( ˆθN−θ0)=⇒N(0, β2∫ 10 (s′(x;θ))2dx)where =⇒ denotes convergence in distribution.14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIP101 102−14−12−10−8−6−4−2Number of SensorsMSE (dB)Amplitude Estimation, SNR=0 dB, β=1, η=0Full Precision1−bit ditheredFigure 1: 1-bit quantization with uniform dithering leads to only logarithmic rate loss.The proof uses elementary facts from probability theoryto first establish the almost sure convergence of TN(θ) toJ(θ ;θ0). The consistency and CLT is then established us-ing techniques similar to those in the analysis of ML esti-mators for i.i.d. observations (see [7, Chapter IV]). We donot present the proof here due to space constraints; it canbe found in the longer version [3]. However, we discuss theintuition behind this result and present simulation results be-low.The main conclusion of the above theorem is that our es-timator only leads to logarithmic loss with respect to the bestpossible rate of 1/N. In other words, there is little loss inperformance with respect to full precision samples. We arenot aware of any work where an estimator of θ0 under a gen-eral signal model based on 1-bit precision samples is ana-lyzed. Hence, we compare our result with an estimator thatcan be derived from the results in [6]. In [6], an estimator ofthe signal s(x;θ0) based on {Y (n)} with a(N) = 0 are given.The estimate of the signal is obtained by local linear filter-ing of the observations followed by a memoryless non-lineartransformation. An estimate of θ0 may be obtained from thesignal estimate using a least-squares fit. Note that this is thesame as solving (3) but using the signal estimate in placeof {cdY (n)}. It can be shown (using a proof similar to thatof Theorem 1) that such an estimate is consistent (in prob-ability) and a CLT holds. The variance in the CLT decayslike 1/N2/3, which is a significant loss compared with therate 1/N for the full precision case. In comparison, we seethat our estimator requires less computation and by choosinguniformly distributed dither, only a logarithmic rate loss isincurred.The reason for the success of our estimator is not immedi-ately clear. To obtain more insight, consider the simple casewhen the observation noise is absent (σt = 0), s(x;θ) = θ ,and θ ∈ [−0.5,0.5]. In the absence of dithering, 1-bit quanti-zation gives only one observation value sign(θ0) and we cando no better than estimating the sign of θ0. However, withdithering (choosing a(N) = 1) the estimate is given byˆθN =1NN∑n=1Y (n) =1NN∑n=1sign(θ0 +Wd(n)) .Since {Wd(n)} are i.i.d. uniform on [−1,1], by the law oflarge numbers we get that ˆθN → θ0 almost surely and a CLTalso holds. In other words, due to dithering, we get a lookat θ0 from a family of 1-bit quantizers and this ‘diversity’helps in obtaining consistency. In the general case, the di-versity provided by dithering causes TN(θ) to converge toJ(θ ;θ0) (as is the case for full precision samples). But fromassumption A3), we know that J(θ ;θ0) = 0 has a unique zerocrossing at θ = θ0. This leads to existence and consistencyof the estimate. The increasing dither magnitude a(N) is re-quired to remove bias introduced in the estimate by the ther-mal noise. We do not present a proof of the main result due tospace constraints, but we illustrate it in Figure 1 with the helpof a simulation. In Figure 1, we plot the mean-square esti-mation error as a function of N for the case when s(x;θ) = θand σ2t = 1. The consistency as well as the logarithmic rateloss compared to full precision case are evident.14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIPWe have not been able to obtain the best possible rate inthe CLT for uniform dither in the general case. However, fora specific example, we identify the rate below.Proposition 1 Consider the problem of estimating θ0 ∈(0,1] using samples {Y (n), 1 ≤ n ≤ N} where,Y (n) = sign(θ0 +a(N)Wd(n)+σtWt(n)){Wd(n)} and {Wt(n)} are as in Theorem 1 above. We notethat in this case TN(θ) = 0 has a unique solution for every Ngiven by,ˆθN =a(N)NN∑n=1Y (n).If a(N)→ ∞ and a2(N)/N → 0, then ˆθN → θ0 in m.s.s. Thebest possible rate for the decay of the mean-square error isobtained by choosing a(N) such thatlimN→∞Na2(N)exp(− (a(N)−θ0)2σ2t)= 1. (4)and in this case√Na(N)(ˆθN −θ0)=⇒ N (µ ,σ2)for constants µ 6= 0 and σ2.We omit the proof here. We, however, note that for achoice of a(N) as in Theorem 1, the limit in (4) is zero. Thusthe rate in Proposition 1 is strictly better than that in Theorem1.3.2 Optimality of Uniform DitheringNow we address the question of whether a different ditherdistribution can improve the rate of convergence even fur-ther. Theorem 2 shows that we cannot do better than uniformdithering for a class of distributions.Theorem 2 Suppose the conditions GN), A1)-A3), DS) andNUD) are true. Then there exists { ˆθN} such that as N → ∞1. P(TN( ˆθN) = 0)→ 1;2. the estimate ˆθN → θ0 in probability;3. the following CLT holds• If a(N) = βN1/(2q), then,√Na(N)(ˆθN −θ0)=⇒ N (µ ,σ2)where µ and σ2 are specified in [3].• If a(N)N−1/(2q) → 0, then(a(N))q−1(ˆθN −θ0)=⇒ δµwhich is the unit mass at µ .• If a(N)N−1/(2q) → ∞, then√Na(N)(ˆθN −θ0)=⇒ N (0,σ2).The main idea in the proof is similar to that in the proof ofTheorem 1. We do not give the proof here.We see that the best rate in the CLT is obtained in thefirst case. In this case the variance in the CLT behaves like1/N(q−1)/q which is strictly worse than the rate in Theorem 1.In particular, if the Gaussian dither is chosen, then q = 3 andwe get the rate is N−2/3. The higher the value of q, the betteris the rate. Note that q → ∞ implies that the dither distribu-tion is getting closer to the uniform distribution. However,the case of uniform distribution is not covered by Theorem2.The existence of an optimal rate for the variance in part 3)of Theorem 2 is a consequence of a basic principle: increas-ing a(N) increases variance but decreases bias. By increasinga(N), we are expanding the family of 1-bit quantizers beingused for observations and this diversity helps in reducing thebias. However, dithering also makes the observations morenoisy and the variance of the estimate increases as a(N) in-creases. Hence there is an optimum rate for a(N), where thegains of diversity due to dithering, and the ill effects of thenoise due to dithering are balanced.4. CONCLUSIONSWe proposed a parameter estimate for a general signal modelbased on dithered 1-bit samples. This estimator is motivatedby the need to design low cost low precision sensors andpossibility of combining measurements from a multitude ofsuch sensors. The main idea is that the full precision samplescan be replaced by suitably scaled 1-bit dithered samples andhence the estimate needs no more computation than the MLestimate in the full precision case. The estimate is consistentfor a wide class of dither distributions. But the uniform dis-tribution leads to the best rate in the CLT amongst a broadclass of dither distributions. Several other properties of theestimate are also established in a longer version of this pa-per ([3]). In particular, due to the hard-limiting operationinvolved in 1-bit sampling, the estimate is also robust and re-mains consistent even for noise with infinite variance. Wehave also analyzed the effect of random sampling, inaccurateknowledge of sampling locations, and unreliable communi-cation of observations in [3].AcknowledgementWe are thankful to the reviewers for their comments and forsuggesting reference [1].REFERENCES[1] I. Bilinskis and A. Mikelsons, “Randomized SignalProcessing,” Prentice Hall International Series, Lon-don, 1992.[2] Z. Cvetkovic and I. Daubechies, “Single-bit Oversam-pled A/D Conversion with Exponential Accuracy in theBit-Rate,” in Proceedings DCC, Utah, USA, March2000, pp. 343–352.[3] Onkar Dabeer and Aditya Karnik, “Signal paramter es-timation with 1-bit dithered quantization,” submittedfor journal publication, 2005.[4] IEEE Journal on Selected Areas in Communications,April 2005, Special Issue on Self-Organizing Dis-tributed Collaborative Sensor Networks.14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIP[5] A. Host-Madsen and P. Handel, “Effects of Samplingand Quantization on Single-Tone Frequency Estima-tion,” IEEE Transactions on Signal Processing, vol.48, no. 3, pp. 650–662, March 2000.[6] E. Masry, “The Reconstruction of Analog Signals fromthe Sign of Their Noisy Samples,” IEEE Transactionson Information Theory, vol. 27, no. 6, pp. 735–745,November 1981.[7] H. V. Poor, An Introduction to Signal Detection andEstimation, Springer, New York, second edition, 1994.[8] J. G. Proakis, Digital Communications, McGraw Hill,New York, third edition, 1995.[9] D. Rousseau, G. V. Anand, and F. Chapeau-Blondeau,“Nonlinear Estimation from Quantized Signals: Quan-tizer Optimization and Stochastic Resonance,” in ThirdInternational Symposium on Physics in Signal and Im-age Processing, Grenoble, France, January 2003, pp.89–92.[10] T. Andersson, M. Skoglund, and P. Handel, “FrequencyEstimation by 1-bit Quantization and Table Look-UpProcessing,” in Proceedings of European Signal Pro-cessing Conference, September 2000.[11] H. L. Van Trees, Detection, Estimation, and Modula-tion Theory - Part I, Wiley, New York, 1968.14th European Signal Processing Conference (EUSIPCO 2006), Florence, Italy, September 4-8, 2006, copyright by EURASIP

Consistent Signal Parameter Estimation with 1-Bit Dithered Sampling

https://zenodo.org/record/53470/files/1568980558.pdf

Consistent Signal Parameter Estimation with 1-Bit Dithered Sampling

Abstract

Similar works

Full text

Available Versions

ZENODO