A closed loop servo approach is applied to the problem of controlling and estimating variance in nonstationary
signals. The new circuit closely resembles but is not the same as, automatic gain control (AGC)
which is common in radio and other circuits. The closed loop nature of the solution to this problem makes this
approach highly accurate and can be used recursively in real time

Moir, T.J.

English

Massey Research Online

Res. Lett. Inf. Math. Sci.,  (2000) 1,  129-140Available online at http://www.massey.ac.nz/~wwiims/rlims/Automatic Variance Control and Variance Estimation LoopsT.J.MoirI.I.M.S., Massey University Albany Campus, Auckland, N.Z.t.j.moir@massey.ac.nzAbstractA closed loop servo approach is applied to the problem of controlling and estimating  variance in non-stationary signals. The new circuit closely resembles but is not the same as, automatic gain control (AGC)which is common in radio and other circuits. The closed loop nature of the solution to this problem makes thisapproach highly accurate and can be used recursively in real time.1 IntroductionIn applications which use adaptive filters, usually some estimate of variance is required if a Least-MeanSquares (LMS) algorithm is used for weight vector estimation [1]. Normally a window or moving window ofdata can be used and the sample variance computed. A finer approach is to update the variance recursively.That is, (assuming large k) for a sample variance 2kσ  of k samples of zero-mean data kyσk2=1kyi2i=1k (1)the recursive equivalent is given by [2]][1 2 12212−−−+= kkkk ykσσσ (2)For stationary signals the recursive variance estimator converges asymtotically  as ∞→k . A similarmethod can be used for estimating the mean recursively [2]. However, if the signal is non-stationary then (2)must be able to track the time-varying variance. The problem is that with large k, equation (2) pays littleattention to new information and ‘switches off’. Sometimes a factor of safety is used and the variancemagnitude is under-estimated to allow for any sudden changes, or if the dynamic range of the signal (as is thecase with speech) is known a worse case upper estimate can be used. In the case of the LMS algorithm if thevariance is under-estimated the algorithm can become unstable due to the step size becoming too large.Conversely if the variance is over-estimated the convergence of the LMS algorithm may well be too slow [1].This basic problem has been recognised in [3] where the authors alter the LMS algorithm to implicitly includeautomatic gain control. It is usual in the literature to use some form of exponential weighting of past data totrack the variance. This facilitates the use of a ‘forgetting factor’. For example it is often  proposed to use2212 )1( kkk yββσσ −+= − (3)130 R.L.I.M.S. Volume 1. Sept. 2000where 0< β <1 is the forgetting factor which controls the bandwidth and the time constant of the first orderrecursive digital filter. For a typical speech signal, using β =0.95 can be made to work although the estimateis not very smooth. Increasing β  gives smoother estimates at the expense of worse tracking.The implicit approach in [3] uses a  forgetting factor approach  similar to the above. Whilst methods like thesecan be made to work for certain applications, they are generally of an ad-hoc nature and are a compromisebetween smoothness of the estimate and tracking ability. The approach used here can be used for accuratetracking of variance or better still to accurately define the variance of a signal with a pre-defined  set-point.The philosophy is similar to that used in radio receivers where an AGC boosts the radio frequency signal to auseful power for later amplification and detection. Hence this approach is proposed as a front end to adaptivealgorithms rather than an implicit change to the LMS algorithm itself, such as has been  proposed in [3]. Anautomatic gain control strategy has been proposed in [4] which improves the performance of an LMS adaptivefilter but it too uses forgetting factors and is highly non-linear.2 Automatic Variance ControlThe automatic variance control (AVC) described here is essentially a form of AGC calibrated to variancerather than amplitude or average power rather than voltage. Whilst it would seem obvious that an AGC maywell suffice it should be recalled that AGCs are largely non-linear and would not result in accurate tracking.Although the AVC uses non-linear elements, its tracking ability is entirely linear with theoretically zerosteady-state error to a step change in variance.The block diagram of the AVC is shown in Figure 1. It comprises a pure squarer, a linear multiplier, asumming junction (with set-point) and an integrator.Figure 1 Automatic Variance Control block diagramThe operation of the circuit is essentially as follows.The input signal )(tf i is assumed to be free of dc. This signal is multiplied by the integrator output y(t)which for a stationary or periodic input signal will be constant. The multiplier output is the output signal withthe pre-defined variance defined by the set-point v(t). Should the input signal power change, the input to theT. Moir, Automatic variance control and variance estimation loops 131squarer will also change momentarily and this is in turn fed into the squarer. When a signal is squared itproduces a dc term plus higher harmonics. The higher harmonics are filtered by the integrator and any changein dc from the squarer output produces either a larger or smaller error from the summing junction which is inturn integrated. The integrator will either ramp up or down to scale the input signal to a pre-defined amountdefined by v(t). A squarer is used within the loop as opposed to a modulus detector normally used in AGCloops since this provides the suitable scaling for variance rather than amplitude.3 Steady-State AnalysisThe analysis follows from the block diagram of Figure 1. The input )(tf i  can be either deterministic orstochastic bust must be non-zero in the long term and free from dc. The integrator output becomesy(t) = K e(t)dt (4)The error signal is defined as)()()( tutvte −= (5)where u(t) is the squarer output which when combined with the multiplier output results in))()(()( 2tytftu i= (6)For any close loop system the gain K must be as high as possible for a given bandwidth. From basic servotheory, assuming K>>1 gives an error 0)( →te  and from (5) and (6))())()(( 2 tvtytfi → (7)However, since )()()( tytiftof = we must have)()(2 tvtfo → (8)This can be interpreted as follows for two classes of input signal.3.1 Deterministic input signalsFor a set-point variance v(t)=1 and input waveform )(tf i consisting of a sine wave of unity amplitude, theoutput of the AVC )(tfo  will have an rms value of unity or and amplitude of 2 . Provided  there is no dcoffset on the input waveform, the output waveform is always scaled so that the rms value of )(tfo followsthe square root of the set-point v(t) (from (8) ). For a sine wave input of frequency uf Hz the squarer output,132 R.L.I.M.S. Volume 1. Sept. 2000from basic trigonometry, produces a dc term and a component at uf2  Hz. Just as with a phase-locked loop,the uf2  term must be sufficiently attenuated by the bandwidth of the loop to avoid distortion. This isachieved by ensuring that the unity gain crossover frequency of the integrator is at least ten times smaller thanthe uf2  component. Hence the bandwidth of the AVC must be chosen to be at least one tenth of the lowestfrequency of interest. A phase-locked loop can operate at much higher bandwidths since the input frequencyis usually in the kHz or MHz region. Clearly for our applications, since the input frequencies are usuallybaseband, the AVC, like an AGC, is a slow acting control loop with a practical bandwidth at most of a fewtens of Hz .3.2 Random input signalsTaking expectations of both sides of (8) gives)]([)]([ 2 tvEtfE o → (9)The variance set-point v(t) is deterministic and clearly )()]([ tvtvE = .Hence from (9) it is seen that the random output signal will be scaled so that its variance tracks the set-point.For example, a zero-mean  white noise input of unit variance and a set-point variance of unity will result in anoutput variance of unity and the statistical characteristics will remain otherwise unchanged. For the same set-point, if the input variance slowly increases or decreases, the output variance will stay at unity.For a time-varying input of say a speech waveform, the AVC will not have sufficient bandwidth to respond toperiods of non-speech. This is a positive result since any amplification of background noise is undesirable asis any distortion of the speech ‘envelope’.4 Estimation of input varianceWith slight modification, the AVC can be used as an accurate recursive method of estimating variance andwill not suffer from the same disadvantages as discussed for equations (1) and (2). This method like all closedloop methods is only limited by the bandwidth of the loop. A fundamental assumption is made here that theintegrator output is statistically independent of the input signal and we can write )]([)]([)]([ 222 tyEtfEtfE io = . (10)This assumption is born out by the fact that the bandwidth of the AVC is small and the integrator smooths outany high frequency components. For stationary input signals the integrator output will be a constant in steady-state and effectively oyty →)( . A similar argument is used in analysis of the weights for an  LMSalgorithm[1].The input variance can now be computed by re-arranging (10) and substituting set-point variance instead of)]([ 2 tfE o  via (9). Then)]([)()]([ 22tyEtvtfE i → (11)T. Moir, Automatic variance control and variance estimation loops 133Which requires the division of the set-point with the mean squared integrator output.  To avoid division,which is computationally time consuming  a further closed-loop system is used as shown in Figure 2 whichconsists of an integrator, a set-point and a multiplier.Figure 2. Variance Estimation loopAn additional squarer is required outside the AVC loop from the integrator which obtains )(2 ty . Theanalysis of the loop of Figure 2 follows closely the method used previously. From the block diagram the  errorsignal is given by)()()()( 222 tytytvte −= (12)Where )(2 ty is the integrator output in Figure 2.Assume the integrator gain K>>1 and the error signal 0)(2 →te . This results in)()()( 22 tytvty → (13)Which is the instantaneous division of the set-point variance with the squared output of the AVC integrator.Taking expectations of (13) results in equation (11) indicating that the integrator output of Figure 2 has avariance equal to the variance of the original input waveform. In steady-state its value is constant and gives adirect reading of variance. For deterministic inputs the variance estimator gives the rms. value squared as anoutput or average power in the waveform. The same restrictions on bandwidth as the AVC do not apply sincethere is no feed-though component, hence the bandwidth of the integrator of this loop is chosen to be as muchas ten times faster than the AVC loop.134 R.L.I.M.S. Volume 1. Sept. 20005 Simulation ResultsThe AVC and variance estimation loops can be implemented either in digital or analogue form. For this workthe loops were implemented in software and several different signal types were tested. For convenience, a set-point of unity was used for all cases. For a sine wave input of unity amplitude and unity set-point  the AVCgave an output of a sine wave with magnitude 1.412 as expected. Figure 3 (a) shows the original sinusoid andFigure 3 (b) shows the AVC output. The integrator output of the AVC is shown in Figure 3 (c) whichconverges to 1.412.  The sinusoid was chosen to have a frequency of 100Hz and the bandwidth of the AVCloop was 10Hz.The ripple is due to 200 Hz feed-through terms caused by the squaring action within the loop.Figure 3. AVC acting on a sinusoid.(a) Input Waveform to AVC )(tf i(b) AVC Output )(tfo(c) Integrator Output y(t)The AVC was then tested on a speech signal. Two different bandwidths were used to illustrate the effect oftoo high a bandwidth. At first, speech sampled at 12kHz was processed with an AVC bandwidth of 0.1Hz.The set-point variance was unity. Figure 4 (a) shows the original speech signal and Fig 4(b) shows the AVCoutput which has been scaled by the AVC so that the average variance is unity across the whole waveform.T. Moir, Automatic variance control and variance estimation loops 135Figure 4. AVC output (bandwidth 0.1Hz)  for a speech waveform.(a) Speech Waveform input to AVC )(tf i(b) Output of AVC )(tfoFor a bandwidth of 10Hz the AVC is then able to compress the speech so as to give unity variance at alltimes. However, the envelope of the speech is ‘flattened’ for this high bandwidth case as is shown in Figure 5.Although it is the lower bandwidth case which would be used in practice, it is worth commenting that theintelligibility  of the speech was unaffected by the AVC for this case.136 R.L.I.M.S. Volume 1. Sept. 2000Figure 5. AVC output (10Hz bandwidth) for a speech waveform.(a) Original Speech Waveform )(tf i(b) AVC Output )(tfoThe variance estimation loop was added to the AVC as in Figure 2 and was initially tested on a sinusoidalinput. Figure 6 (a),(b) and (c) shows the original 100Hz sinusoid, the AVC output and the variance estimatoroutput respectively. The variance converges to 0.5 which represents the rms. value of the input signal squared.Tests were also carried out on other types of periodic waveforms.T. Moir, Automatic variance control and variance estimation loops 137Figure 6 Variance Estimation for a sinusoidal input.(a) Signal Input to AVC )(tf i(b) AVC Output )(tfo(c) Variance Estimator Output )(2 tyFor the unity white noise input waveform shown in Figure 7 (a) , the variance estimation loop output (Figure7 (c) )  converges to unity and the AVC output (Figure 7 (b))  remain unchanged.138 R.L.I.M.S. Volume 1. Sept. 2000Figure 7. Variance estimation for white noise unit variance.(a) White Noise input to AVC )(tf i(b) AVC Output )(tfo(c) Variance Estimator Output )(2 tyFinally if the bandwidth of the loops are taken as 10Hz, then for a speech signal it is possible to track itsvariance with time. Figure 8 (a) shows the original speech waveform, Figure 8 (b) shows the AVC flattenedoutput and Figure 8 (c) shows the variance estimator output. Although the AVC output has the ‘flattened’form discussed in the earlier example, this is of no consequence if it is tracking ability of the variance whichis primarily required rather than setting the total average variance to a pre-defined set-point.T. Moir, Automatic variance control and variance estimation loops 139Figure 8. Tracking the variance of a speech waveform.(a) Speech Waveform input to AVC )(tf i(b) AVC Output )(tfo(c) Variance Estimator Output )(2 ty6 ConclusionsA new closed-loop circuit termed Automatic Variance Control has been presented which has the ability toalter the variance of a signal to a pre-defined value on a sample by sample basis rather than relying on batch-data type computations. A second control loop which interfaces to the AVC has the ability to estimate thevariance of the original input signal and track it accurately with time. Several examples were shown whichillustrate the operation of both loops. The new technique will have applications in areas of adaptive signalprocessing which are variance sensitive, such as LMS adaptive filters.140 R.L.I.M.S. Volume 1. Sept. 20007 References[1]  Haykin,S,” (1986) Adaptive filter theory”,Prentice Hall Englewood Cliffs,New Jersey[2] Young, P.C. (1984) Recursive Estimation and Time Series Analysis: An Introduction. Springer Verlag:Berlin.[3] Shan,T.J and Kailath,T, (1988), Adaptive algorithms with an automatic gain control feature, IEEETransactions on Circuits & Systems, vol.35 no.1 ,pp122-127[4] Tapia J. Kuo S.M., An optimized automatic gain controller for real time recursive LMS adaptive filters,1990 IEEE International Symposium on Circuits and Systems (Cat.No.90CH2868-8). IEEE. 1990, pp.3154-7vol.4. New York, NY, USA.

Automatic variance control and variance estimation loops

https://mro.massey.ac.nz/bitstream/10179/4381/1/Automatic_Variance_Control_and_Variance_Estimation_Loops.pdf

Automatic variance control and variance estimation loops

Abstract

Similar works

Full text

Available Versions

Massey Research Online