We consider the quantization of a special class of non

bandlimited signals, namely the class of discrete time signals that can be recovered from their decimated version. Similar to sigma-delta modulation ideas, we show that we can obtain a great reduction in the quantization noise variance due to the oversampled nature of these signals. We then consider noise shaping by optimizing a pre- and post filter around the quantizer and develop a closed form expression for the coding gain of the scheme under study. The use of

an orthonormal filter bank as a sophisticated quantizer is

investigated and several examples are provided

Tuqan, Jamal

Vaidyanathan, P.P.

English

Caltech Authors - Main

Optimum Quantization of a Class of Non Bandlimited Signals 
Jamal Tuqan and P. P. Vaidyanathan 
Department of Electrical Engineering 136-93 
California Institute of Technology 
Pasadena, CA 9 1125 
E-mail : tuqan@systems.caltech.edu, ppvnath@sys.caltech.edu 
Abstract In this paper, we restrict our analysis to the single band 
We consider the quantization of a special class of non 
bandlimited signals, namely the class of discrete time signals 
that can be recovered from their decimated version. Similar 
to sigma-delta modulation ideas, we show that we can obtain 
a great reduction in the quantization noise variance due to 
the oversampled nature of these signals. We then consider 
noise shaping by optimizing a pre- and post filter around 
the quantizer and develop a closed form expression for 
the coding gain of the scheme under study. The use of 
an orthonormal filter bank as a sophisticated quantizer is 
investigated and several examples are provided. 
1. Introduction 
Walter [l] showed that, under some conditions, a 
class of non bandlimited continuous time signals can 
be reconstructed from uniformly spaced samples even 
though aliasing occurs. Vaidyanathan and Phoong [2], 131 
developed the discrete time version of Walter's result from 
a multirate digital filtering perspective. In specilic, they 
introduced the class of non bandlimited signals that can be 
modeled as the output of a single interpolation filter (single 
band model) as in Fig. 1.1 or as the output of the more 
general multiband model of Fig. 1.2. It can be shown 
that this class of non bandlimited signals can be recovered 
from its decimated version. As a quick example, assume 
that z(n) is modeled as in Fig. 1.1 and consider z ( M n ) ,  
the M-fold decimated versions of z(n). If F(ej") is a 
Nyquist(M) filter [4], then, z ( M n )  is equal to y (n) and we 
have the relation ~ ( n )  = rc(kM)f (n  - kM). In other 
words, z(n) is completely defined by the samples z ( M n )  
even though the Elter F(ej") is not necessarily ideal. More 
elaborate "sampling theorems" can be developed for Fig 1.1 
and Fig. 1.2. The details can be found in [2] and [31. 
k 
model of Fig. 1.1 and consider the- efficient quathation 
of this class of non band-limited signals. To motivate such 
a study, consider the schematic of traditional sigma delta 
modulation shown in Fig. 1.3 where the box labeled G! 
represents a PCM quantizer. The signal z(n) is assumed to 
be bandlimited (oversampled). After the quantization, the 
ideal low pass filter on the right chops off the noise in the 
stopband but does not change the signal component. The 
signal power is the same whereas the noise power decreases 
proportionally to the oversampling ratio. For bandlimited 
signals, we can therefore quantize the signal to very few bits, 
perhaps even one bit if we oversample the signal enough. In 
addition, we can generate a further decrease in the noise 
power by introducing noise shaping in the signal band, 
as for example in sigma-delta modulators, to allow higher 
resolution quantization of bandlimited signals. 
The schematic description of Fig. 1.3 shows that we 
can take advantage of a signal model like Fig. 1.1 even 
though z(n) is not bandlimited. Thus, consider Fig. 1.4 
where the finite order filter F(ej") is assumed to be an 
optimum compaction filter. In specific, the filter maximizes 
the variance of its output signal under the constraint that its 
magnitude squared response IF(ej") l2 is Nyquist(M), that 
is, (IF(eJU">I") J.M= 1. Theassumptionis motivatedby the 
fact that this particular choice of filters minimize the mean 
square reconstruction error between a signal, say z (n) ,and 
its approximation modeled as in Fig. 1.1 [51. With this 
last assumption, it can be shown that the signal k(n)  in 
Fig. 1.4 is equal to ~ ( n )  in the absence of the quantizer. 
The entire scheme of Fig. 1.4 behaves similar to Fig. 1.3, 
except that the low pass filtering is multirate and non Ideal. 
Thus, generally speaking, if anon bandlimited signal can be 
reconstructed from its samples z ( M n )  because it satisfies a 
model like Fig. 1 .l, then, a low precision quantizer should 
allow us to produce a high precision version ?(n). 
The quantization advantage offered by Fig. 1.4 can be 
Work supported in parts by Office of Naval Research grant N00014-93-1-0231, Tektronix, Inc., and Rockwell International. 
1396 
1058-6393/97 $10.00 0 1997 IEEE 
useful, for example, in the following realistic engineering 
scenario. Suppose x(n)  is generated at a point where we 
cannot afford very complex signal processing (e.g., in deep 
space) and needs to be transmitted to a distant place (e.g., 
earth station). If we have the knowledge that z (n)  admits 
a satisfactory model like Fig. 1.1, we can compress it 
using a very simple low pass filter P(ej") with one or two 
multipliers and then quantize the output before transmission. 
The post filter l/P(ej") and the expensive multirate filter 
are at the receiver end, where the complexity is acceptable. 
In the sequel we shall find an expression for the theoretically 
best P(ej") without constraint on order. This will give 
an upper bound on the gain obtainable with a practical 
inexpensive P(ej"). 
2. Exploiting the signal model 
Consider the set up shown in Fig. 1.4 where the input 
signal x(n) satisfies the the single band model of Fig. 
1.1. Our assumptions are as follows : the driving signal 
y(n) in Fig. 1.1 is a zero mean wide-sense stationary 
( W S S )  random process. The filter F(ej") is FIR and has 
the property that lF(ejW)l2 is Nyquist(M) for the reasons 
described above. Because the model filter F(ej") is not 
ideal, the input signal x(n) in Fig 1.4 is a cyclo wide-sense 
stationary signal of period M ( (CWSS)M)  [6].  The box 
labeled Q represents a scalar uniform (PCM) quantizer and 
is modeled as an additive zero mean white noise source 
q(n).  We will design the quantizer Q as follows : since 
the input to the quantizer ~ ( n )  is a (CWSS)M process, 
its variance o:(n) is a periodic function of n with period 
M. Define a$ to be the average variance of x(n) ,  i.e., 
a: = - 02 (n). Then, choose the fixed step size A 
M 
in the uniform quantizer such that the quantization noise 
variance ai is directly proportional to the variance of the 
quantizer input x(n). In specific, we design the uniform 
quantizer such that the following relation holds 
M-1 
n=O 
where ai is the quantization noise variance, c is a constant 
that depends on the statistical distribution of x(n)  and the 
overflow probability, and ai is the average variance of the 
quantizer input. The above relation is justilied for a PCM 
quantizer using 3 (or more) bits per sample (see chapter 4 in 
[7]). The next theorem gives an expression for the average 
mean squarederror & = - M 
Theorem 2.1. Consider the scheme of Fig. 1.4 under 
the above assumptions. The average mean square error 
1 
E is equd to -03. 
M 
M-1 
A 1  E{P(n) - ~ ( n ) } ~ .  
n=O 
The proof can be found in [8]. The quantization noise 
variance a: obtained by directly quantizing ~ ( n )  as shown in 
Fig. 2.1 is now reduced proportionally to the oversampling 
factor M. The signal variance a," on the other hand did 
not change. By expressing the interpolator M in the form 
2r,  we can immediately see that we can get the same 
quantitative advantage of the oversampling PCM technique, 
namely, an increase in S N R  by 3 db for every doubling 
of the oversampling factor. For example, if M = 2, we 
get an SNR increase of 3 db whereas if M = 4, the 
S N R  increment is by 6 db. The result of theorem 2.1 
can be intuitively explained. The signal ~ ( n ) ,  modeled 
as in Fig. 1.1 is oversampled and therefore, contains 
redundant information in the form of an excess of samples. 
It is by quantizing these extra samples that we obtain the 
reduction in the quantization noise variance (equivalently 
in the mean square error). We are therefore effectively 
quantizing with a higher number of bits per sample. This 
trade off, between the quantization noise variance (effective 
quantizer resolution) and the sampling rate is the underlying 
principle of oversampled A/D converters. 
A consequence of the previous results and discussion is 
then the natural question: what if the discrete time filtering 
of the oversampled signal is not a major burden ? If we 
know that x(n)  can be modeled quite accurately by the filter 
F(ej") of Fig. 1.1, we can filter and downsample z(n) to 
obtain y(n) as shown in Fig. 2.2. We can then in principle 
quantize the decimated signal y(n) with 6 = Mb bits per 
sample. This situation is equivalent to fixing the bit rate 
(number of bits per second) to be equal to b in order to 
trade quantization resolution with sampling rate. At this 
point, we will however assume that the goal is to actually 
obtain a reduction in the bit rate. To achieve this, we fix 
the number of bits per sample b to be equal to b. Since the 
quantizer resolution did not increase, the quantization noise 
variance should not differ from the direct quantization case 
of Fig. 2.1. This last statement is verified formally in the 
next theorem. 
Theorem 2.2. Consider the scheme of Fig. 2.2. With 
a fixed number of quantizer bits b, the average mean 
square error E is equd to ai, where ai is the noise 
variance obtained from directly quantizing x(n) using 
b bits. 
The proof can be found in 181. 
3. Noise Shaping 
Following the philosophy of sigma-delta modulators, we 
would like now to perform noise shaping with the hope of 
achieving a further reduction in the average mean square 
error. To accomplish this, we propose using LTI pre- and 
post filters around the PClM quantizer as shown in Fig. 3.1. 
The goal is to optimize the filter P( ej") such that the average 
1397 
m.s.e at the output of Fig. 3.1. is minimized. At this point, 
no order constraint is imposed on the filters and non causal 
solutions are accepted. 
Following (2.1), the quantizer noise variance in this 
case is given by CY: = c ~ - ' ~ c Y , ~  where CY: is the average 
variance of the process z(n).  We emphasize that z(n) 
is a (CWSS)M process since the output of a linear time 
invariant filter driven by a (CWSS)M process is also 
(CWSS)M [6]. It is then possible to express CY: in terms of 
the prefilter P(ej") and the so called average power spectral 
density of theprocess z(n), denotedby &,(ej"), as follows 
The average power spectral density is a familiar concept 
that arises when stationarizing a (CWSS)M process 191 and 
satisfies the well known properties of the power spectrum of 
a WSS process. It is defined to be the discrete time Fourier 
transform of the time averaged autocorrelation function . M-1 
scheme of 3.1 is 
where &w is the half whitening coding gain [7] of the 
WSS process y(n). 
The factor M in (3.3) is again due to the oversampled 
nature of the signal ~ ( n ) .  It is interesting to note that 
the noise shaping contribution to Goopt in (3.3), which we 
denote by Bhw, is exactly the coding gain we would obtain 
by half whitening the WSS process p(n) in the usual way 
[7] .  By appealing to the Cauchy Schwartz inequality, we 
can show that 6hw 2 1 with equality iff the power spectral 
density Sy,(ejw) is a constant, i.e., y(n) is white noise. 
Therefore, for the particular system of Fig. 3.1, we will not 
get additional coding gain by noise shaping if the driving 
WSS process ~ ( n )  in Fig. 1.1 is white noise. 
1 
Example 6.1. Case o f a  MA(I) process y(n). Assume 
that the input z(n) is modeled as in Fig. 1.1 where 
the upsampler M = 2, the filter F(ej"> is b e  Optimum 
orthonormal FIR filter of length two given by - (1 + z-l) 
and the driving WSS signal y(n) is a zero mean gaussian 
MA( 1) process with an autocorrelation sequence in the form 
A Z Z ( k )  given by - M E [ z ( n ) ~ * ( n  - I C ) ] .  n=O 
Theorem 3.1. Consider the scheme of Fig. 3.1 U- 
der the same assumptions of section 11. The opti- 1 
Jz mum prefilter P(ej") that minimizes the average mean 
square reconstruction error has the following magni- 
tude squared response: 
k = 0. 
otherwise. 
+ 8' k = 1, -1. 
The proof of equations (3.1) and (3.2) can again be found 
in [8]. A number of observations Should be made at this 
point. First, the opthum filter is not unique since the phase 
response is not specified, Second, the above derivation 
assumes that the input spectrum SZZ (ej") # o for all w. 
The assumption is areasonable one becayse ~ ( n )  is assumed 
to be non bandlimited and therefore S,,(ejy) cannot be 
identically zero on a segment of [0,2?r). If S,,(ej') has 
an isolated zero for some w,  then, the resulting prefilter will 
It is Well k"I that a MA(1) Process has to have 
I&(') I 5 1/42 to ensue that the power spectral density is 
&,I, (0) 
indeed non negative. we therefore restrict 8 t0 be between 
-1 and 1. The power spectrum of the MA(1) process is 
given 
(3.4) 
e 
Sy,(eju) = 1 - 2- (1 + 8 2 )  Cos(W)  
have a zero on the unit circle and is therefore unstable. In 
any case, a practical system would use only a stable rational 
approximation of the ideal solution. Using (3.2), we can 
Substituting (3.4) in (3.3) and after some manipulations, the 
coding gain of the scheme of Fig. 3.1 can be expressed as: 
(3.5) 
(1 + 8 2 )  derive an interesting expression for the coding gain of the 
scheme of Fig 3.1. The coding gain of a quantization scheme 
is defined to be the ratio &diTect/&min where &direct is the 
mean square error obtained by quantizing z (n) directly with 
b bits as shown in Fig. 2.1 and &min is the minimum mean 
squared error obtained by using optimum pre and post filters 
around the quantizer under a fixed bit rate assumption. 
Theorem 3.2. With the optimum choice of the p r e  
and post filters, the coding gain expression for the 
+ Q)p(~m/(l+ WIZ Gopt = 2 
where E(.) is the complete elliptic integral of the second 
kind. The plot of the coding gain as a function of 8 is shown 
inFig. 3.2. 
Example 6.2. Case of an A R ( I )  process y(n). With 
the same assumptions as in example 6.1, let the driving 
signal y(n) be a zero mean gaussian AR(1) process with an 
1398 
autocorrelation sequence in the form R,,(k) = plkl where 
p is between 0 and 1. The power spectrum of the AR(1) 
process is 
(3.6) 
1 - p 2  w - S,de - 1 + p 2  - 2pcos(w) 
Substituting (3.6) in (3.3) and simplifying, the coding gain 
for the scheme of Fig. 3.1 can be expressed as follows: 
- 2 
(3.7) 
where K ( p )  is the complete elliptic integral of the first kind. 
The plot of the coding gain as a function of p is shown in 
Fig. 3.3. 
4. Using an orthonormal filter bank 
Since the signal model ~ ( n )  is (CWSS)M by construc- 
tion, restricting ourselves to linear time invariant noise shap- 
ing filters and quantizers is a loss of generality. The opti- 
mum noise shaping filters for such processes should be lin- 
ear periodically time varying (LPTV)M filters surround- 
ing a (PTV)M quantizer. This is equivalent to M LTI 
analysis and synthesis filters surrounding A4 time invari- 
ant uniform scalar quantizers. We can further impose the 
perfect reconstruction condition in the absence of quantiza- 
tion by confining ourselves to the class of M LTI analysis 
and synthesis filters satisfying the biorthogonality condi- 
tion: (Pk(ej")Q,(ej")>l J.M= 6(m-k)  forallk,m. The 
goal is then to find the set of M analysis and synthesis fil- 
ters, Pk (ej") and Q k  (ej"), that minimize the average mean 
square error at the output due to quantization noise. Be- 
cause the general (LPTV)M problem is difficult to track 
analytically, we will restrict ourselves to the special class 
of orthonormal (LPTV)M filters, i.e., filters satisfying the 
following properties: Q k ( e j w )  = &(ej"') for each k and 
(Pk(ej")Fm(ej"))l J.M= d(m-k) forallk,m. Thegoalis 
to jointly allocate the subband bits bk under the fixed bit rate 
constraint b = - bk and optimize the filters Pk(ej") 
in order to minimize the average m.s.e. 
Theorem 4.1. Consider the scheme of Fig. 4.1 
under the above assumptions. The synthesis section 
of the optimum orthonormal filter bank {Pk(e j" )}  
corresponds to  choosing one of the filters, say Po(ej") 
to be equal t o  F(ej") and the remainingfilters &(e'"'), 
k = 1 , .  . . , M - 1, to be orthogonal to &(ej"). In this 
case, the optimum orthonormal filter bank reduces to 
Fig. 2.3 where the quantizer Q is now allocated M b  
bits per sample. 
The proof can be found in [8]. The result of Theorem 
4.1 is very intuitive and somewhat expected: Decimate the 
1 M-l 
k=O 
M 
oversampled signal ~ ( n )  according to its model and then 
quantize the decimated signal y (n,) in Fig. 2.2 with 6 = M b  
bits per sample. This amounts to fixing the bit rate (number 
of bits per second) in order to trade quantization resolution 
with sampling rate. It is interesting though to see that this 
very intuitive scheme is equivalent to using an optimum 
orthonormal filter bank as a sophisticated quantizer to the 
input ~ ( n ) .  Using (2.1), the coding gain expression can be 
derived and is equal to 22b(M-1) .  The coding gain depends 
on the bit rate b and can be quite large for moderate values 
of M and b. 
References 
[l]  Walter, G.G., "A sampling theorem for wavelet subspaces", 
IEEE Trans. on Information Theory, pp. 881-884, March 1992. 
[2] Vaidyanathan, P.P. and Phoong, S., "Reconstruction of 
sequences from non uniform samples", ISCAS Proc., pp. 601- 
604, Seattle 1995. 
[3] Vaidyanathan, P.P. and Phoong, S., "Discrete time signals which 
can be recovered from samples", ICASSP Proc., pp. 1448-1451, 
Detroit 1995. 
[4] Vaidyanathan, P.P., Multirate systems andfilter banks, Prentice 
Hall, Inc., Englewood Cliffs, 1993. 
[SI Tsatsanis, M.K. and Giannakis, G.B.,"Principal component 
filter banks for optimal multiresolution analysis", IEEE Trans. on 
Signal Processing, pp. 1766-1777, Vol. 43, August 1995. 
[6] Sathe, V.S. and Vaidyanathan, P.P., "Effects of multirate 
systems on the statistical properties of random signals", IEEE 
Trans. on Signal Processing, pp. 131-146, Vol. 41, January 1993. 
[7] Jayant, N.S. and Noll, P. Digital coding of wavefonns, Prentice 
Hall, Inc., Englewood Cliffs, New Jersey, 1984. 
[8] 'hqan, J. and Vaidyanathan, P.P., "Sigma-delta modulators like 
techniques for quantizing a class of non-bandlimited signals", to 
be submitted. 
[9] Gardner, W.A. and Franks, L.E., "Characterization of cyclosta- 
tionary random processes", IESEE Trans. on Information Theory, 
vol. 21, No. 1, pp.4-14, January 1975. 
?An) *F-+ 44 
Fig. 1.1. The single band model 
1399 
x(n) P ( P )  t+J+'/p(ej")+ 
Quantizer 
44 F (e'? t+++J+ F (e'") 
Fig. 1.4. Exploiting the signal model in quantization. 
h 0 
=/A4 
z(n) + 5(n) 
Fig. 2.1. Direct quantization of ~ ( n ) .  
Fig. 3.2. Coding gain curve as a function of theta 
with M = 2 and y(n) is an MA(1) process. 
1 
.E 1 
-c) 
U 
Iz 
a 
CJ) 
m 
II: 
U 
.- 
.- 
8 
L 
0 0.5 1 
Fig. 3.3. Coding gain curve as a function of rho 
with M = 2 and y(n) is an AR(1) process. 
Fig. 4.1. Using an orthonormal filter bank as a sophisticated quantizer. 
1400 


Optimum quantization of a class of non-bandlimited signals

A sampling theorem for wavelet subspaces&quot;,

Characterization of cyclostationary random processes&quot;,

Digital coding of wavefonns,

Discrete time signals which can be recovered from samples&quot;,

Effects of multirate systems on the statistical properties of random signals&quot;,

Multirate systems andfilter banks,

Reconstruction of sequences from non uniform samples&quot;,

Sigma-delta modulators like techniques for quantizing a class of non-bandlimited signals&quot;, to be submitted.

http://authors.library.caltech.edu/10459/1/TUQasilo96.pdf

Optimum quantization of a class of non-bandlimited signals

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main

Caltech Authors - Main