We consider the problem of optimal, data-dependent zerotree design for use in weighted universal zerotree codes for image compression. A weighted universal zerotree code (WUZC) is a data compression system that replaces the single, data-independent zerotree of Said and Pearlman (see IEEE Transactions on Circuits and Systems for Video Technology, vol.6, no.3, p.243-50, 1996) with an optimal collection of zerotrees for good image coding performance across a wide variety of possible sources. We describe the weighted universal zerotree encoding and design algorithms but focus primarily on the problem of optimal, data-dependent zerotree design. We demonstrate the performance of the proposed algorithm by comparing, at a variety of target rates, the performance of a Said-Pearlman style code using the standard zerotree to the performance of the same code using a zerotree designed with our algorithm. The comparison is made without entropy coding. The proposed zerotree design algorithm achieves, on a collection of combined text and gray-scale images, up to 4 dB performance improvement over a Said-Pearlman zerotree

Effros, Michelle

English

Caltech Authors - Main

Zerotree Design for Image Compression: 
Toward Weighted Universal Zerotree Coding 
Michelle Effros * 
Department of Electrical Engineering (1 36-93) 
California Institute of Technology 
Pasadena, California 9 1 125 
effros @caltech.edu 
Abstract 
We consider the problem of optimal, data-dependent ze- 
rotree design for  use in weighted universal zerotree codes 
for image compression. A weighted universal zerotree code 
(WUZC) is a data compression system that replaces the 
single, data-independent zerotree of Said and Pearlman 
with an optimal collection of zerotrees for good image cod- 
ing performance across a wide variety of possible sources. 
We briefly describe the weighted universal zerotree en- 
coding and design algorithms but focus primarily on the 
problem of optimal, data-dependent zerotree design. We 
demonstrate the pe$ormance of the proposed algorithm by 
comparing, at a variety of target rates, the pe~ormance 
of a Said-Pearlman style code using the standard zerotree 
to the pe$ormance of the same code using a zerotree de- 
signed with our algorithm. The comparison is made with- 
out entropy coding. The proposed zerotree design algo- 
rithm achieves, on a collection of combined text and gray- 
scale images, up to 4 dB performance improvement over a 
Said-Pearlman zerotree. 
1 Introduction 
Data compression systems are redundancy removing 
systems. As a result, assumptions or a priori knowledge 
about the types of data to be compressed play a crucial role 
in the design of most compression algorithms. Sometimes, 
this role is explicit - as in the design of a vector quantizer, 
where the designer gathers a set of training data and uses 
that data in the design of an appropriate collection of code- 
words. In other algorithms, the assumptions employed by 
the code’s designer may be less obvious, but nonetheless 
implicit in the code structure. JPEG’s use of the DCT, for 
example, suggests an implicit assumption that the images 
to be coded will be from the class of “smooth, natural” im- 
ages for which the DCT serves as an effective decorrelating 
transform. 
*This material is based upon work supported by NSF Grant No. MIP- 
9501977. 
Like these other algorithms, the zerotree algorithms of 
Shapiro [ 11 and Said and Pearlman [2]  rely upon a variety 
of assumptions about the types of data to be compressed. 
In particular, the designers assume first that most of the en- 
ergy in the images to be compressed will be found in the 
lower frequency bands of those images’ wavelet decom- 
positions (another smooth image assumption). Second, the 
designers assume that the high energy values that do appear 
in higher frequency bands will usually be predictable by 
high energy values in the corresponding coefficients from 
lower frequency bands. In terms of compression perfor- 
mance, the first of these properties plays the greater role. 
That is, while the performance of zerotree codes on data 
sets that meet the first but not the second condition is still 
quite good, zerotree codes perform poorly on data sets that 
meet only the second of our two conditions. 
When the assumptions upon which we build a com- 
pression system fail, performance suffers. Just as a vector 
quantizer designed for one source and operated on another 
will not achieve the optimal performance on the source in 
operation, a zerotree code used to compress an image that 
has most of its energy in the high frequency bands, for ex- 
ample, will yield poor performance, and can even lead to 
data expansion rather than compression. 
We are interested in compression systems that are used 
to compress a variety of different images. To guarantee 
good performance across a wide class of possible data 
types, we can replace any single code with a family of 
codes. Given a collection of possible source codes, the 
optimal performance achievable with that collection is the 
performance associated with compressing each data block 
with the code from our collection that best matches the 
statistics of that block. We will call a code that achieves 
this performance an omniscient code, since both the en- 
coder and decoder of such a code must somehow divine the 
optimal code for each data block and use that knowledge 
to switch appropriately among the codes in the collection, 
Unfortunately, practical source codes cannot be omni- 
0-8186-8183-7/97 $10.00 0 1997 IEEE 
6 16 
scient. In a real compression system, the encoder has ac- 
cess to the uncoded data, and thus can effectively choose 
the best strategy for the data to be coded. The decoder, 
however, has no basis upon which to choose the optimal 
code, and thus a real decoder cannot be omniscient. As a 
result, real data compression systems that optimally switch 
strategies to match the statistics of an unknown or time- 
varying data source must use overhead bits to describe to 
the decoder which code will 6e employed on each data 
block. We call such a switched coding strategy a two-stage 
source code since the data is described in two sections or 
stages. In the first-stage description, the encoder describes 
a code from its collection. In the second-stage description, 
the encoder describes the data using the code described in 
the first stage. 
A universal source code is a single data compression 
algorithm that asymptotically achieves the optimal perfor- 
mance on every source in some broad class of possible 
sources. Most universal source codes in the literature can 
be described in the above two-stage coding paradigm. Uni- 
versal source coding theory has contributed both to the un- 
derstanding of optimal coding performance and to the de- 
sign of practical codes for image compression. Using this 
theory, we gain insight into the design of collections of a 
wide variety of types of codes including vector quantizers, 
quantization matrices for JPEG-style codes, and transform 
codes [3,4, 5 , 6 ,  71. 
We here consider the use of universal source coding 
techniques to design a universal zerotree code, a collection 
of zerotree codes that achieves, on every source in some 
broad class of possible sources, performance approach- 
ing the performance that would be achieved if the optimal 
zerotree were employed on every data block. While [ l ]  
and [2] provide examples of good zerotrees for the set 
of natural images, they do not describe a general method 
for designing zerotrees for other classes of images. Thus 
in order to develop a weighted universal zerotree code 
(WUZC), we must understand the design and operation of 
both two-stage codes and single zerotree codes. Section 2 
contains a description of the optimal WUZC encoding and 
design algorithms. Section 3 treats the design of a single 
zerotree. Section 4 contains experimental results. 
2 Weighted Universal Zerotree Coding 
In traditional zerotree coding, we use a wavelet decom- 
position and quadtree to impose an ordering on a collec- 
tion of image coefficients. At the root of this quadtree are 
placed coefficients from the lowest frequency image sub- 
band, below them are the next higher bands, below them 
even higher bands, and so on. Image coefficients within 
this structure are described one band at a time, with the 
encoding process traveling only as deep in the given tree 
617 
structure as necessary to describe all values above an oc- 
tavely decreasing threshold. The zerotree description is 
most efficient when the smallest subtree containing all co- 
efficients below a given threshold is as shallow as possible, 
or equivalently, when that smallest subtree contains as few 
below-threshold nodes as possible. 
We use 2 = ( X I ,  ..., x1) E X' to represent an 1- 
dimensional data vector of wavelet packet coefficients. 
Given an encoding algorithm like the one described in [2], 
associated with any zerotree description Z is a quantizer 
C = p o a with encoder a : XI -+ S and decoder p : S -+ 2' 
that together map the input space X' of possible data vec- 
tors to the output space of possible reproductions by 
way of a binary prefix code S. (We will associate with each 
zerotree a target final rate, and thus assume that a(.'), or 
more explicitly la(.')l is a constant for any x" E x'.) Let 
4 x ' , Z )  = 4.',P(a(.'))) 
r(x',Z) = la(x') l  
be the distortion and rate respectively achieved by quantiz- 
ing x' with zerotree 2 and the Said-Pearlman encoder. 
We next consider a collection Zl,Zz, ..., ZM of ze- 
rotrees. Using the quantization interpretation of a two- 
stage weighted universal code [3, 41, we consider this col- 
lection to be a codebook of zerotrees. Thus we define a 
"first-stage quantizer" f3 o &, with encoder : X N  -+ s and 
decoder f3 : s -+ C. The first-stage quantizer maps the in- 
put space of possible data blocks to the output space of 
possible zerotrees by way of the prefix code S.  We here 
assume that N is a multiple of 1. The first-stage encoder 
chooses for each N-block a single zerotree. We then use 
the chosen zerotree to encode each of the 1-vectors in x!. 
Using the above first-stage quantizer, the total distor- 
tion associated with encoding data block x with zerotree 
B(a(.P)) is 
and 
NI' 
42,P(&(.sJ))> = C d ( d , P ( & ( x " .  
i= 1 
Likewise, the rate associated with encoding data block 
with zerotree p(&(#)) is 
NI' 
.(.N,P(W))) = Ia(.N)I + .(d>&(.N)>, 
i= 1 
where the rate associated with the choice of a particular 
zerotree includes both the rate needed to describe the cho- 
sen zerotree and the rate used to describe the data given the 
zerotree described in the first stage. 
Using a Lagrangian in order to minimize the distortion 
subject to a constraint on the rate, the optimal first-stage 
encoder a* for a given p is 
for every A?. Thus the optimal first-stage encoder chooses 
the code in its collection that achieves the best average 
Lagrangian performance in encoding the subvectors of a 
given vector ,I?. We call the optimal first-stage encoder a 
nearest neighbor encoder. 
for a given first-stage 
encoder Ci satisfies 
p*(~)  = argminE Z [d(XN,ij(F))+hr(X,p(F))l&(xN) =SI 
The optimal first-stage decoder 
for every 5 E S.  In this case, the optimal zerotree would be 
the zerotree that yields the lowest Lagrangian performance 
at slope 3, on the distortion-rate curve. We call the process 
of designing the optimal first-stage decoder decoding to the 
centroid. Unfortunately, optimal zerotree design remains 
an open problem. We consider a new iterative technique 
for zerotree design in Section 3.  
The code design algorithm employs an iterative descent 
technique to minimize the expected Lagrangian perfor- 
mance. We initialize the algorithm with an arbitrary pre- 
fix code s and collection {P(f) : F E s} of zerotrees. Each 
iteration proceeds through three steps. 
1 Nearest Neighbor Encoding. 
Optimize the first-stage encoder Ct for the given first- 
stage decoder p and prefix code s. 
Optimize the first-stage decoder 0 for the newly re- 
designed first-stage encoder and the given first-stage 
prefix code S. 
Optimize the first-stage prefix code for the newly re- 
designed first-stage encoder i? and decoder p. The opti- 
mal prefix code 9 for a given first-stage encoder & and 
decoder p is the entropy code matched to the probabili- 
ties P{Ct(XN) = F}, for which the ideal codelengths are 
Each step of the algorithm decreases the (non-negative) ex- 
pected Lagrangian performance, and thus the algorithm is 
guaranteed to converge. 
While the system is designed for a target rate (controlled 
by changing A), it retains its embedded binary description 
and can therefore be used for early reconstructions and run 
past its target rate if desired by the end user. Thus the code 
is optimized for a target rate, but achieves good perfor- 
mance for a wide array of possible rates. 
3 Zerotree Design 
Given the wavelet (or wavelet packet) decomposition 
of a particular data set, we wish to order the coefficients 
2 Decoding to the Centroid. 
3 Optimizing the Prejix Code. 
\PI= -logP{Ct(XN) = 57). 
into the tree that best reflects the statistics of the given 
data. Tree design incorporates minimum threshold de- 
sign, coefficient ordering, and tree branching rate. (The 
Said-Pearlman zerotree uses a quadtree structure and thus 
a branching rate of 4). Optimal zerotrees are expected to 
be data and rate dependent. 
To design the optimal threshold, ordering, and branch- 
ing rate, we must understand the tree properties that yield 
good coding performance. As discussed earlier, the suc- 
cess of zerotree codes relies primarily on two factors: the 
use of the shallowest possible tree for a given minimal 
threshold, and the use of a tree in which high energy coef- 
ficients deep in the tree are well-predicted by correspond- 
ing high energy coefficients closer to the tree’s root. No- 
tice that the need for the second factor kicks in when the 
first factor fails. That is, if all of the energy of the source 
is contained in the shallower levels of the associated ze- 
rotree, then the second factor becomes less significant. Re- 
call also that the second factor alone does not yield good 
compression performance. That is, a tree that contains a 
lot of energy deep within its structure will not yield good 
performance even when that energy is perfectly predicted 
by lower levels of the tree. In designing our zerotrees, we 
will therefore concentrate on the first of these two factors. 
For simplicity, we do not include entropy coding in our ze- 
rotree codes. 
The zerotree design algorithm is an iterative descent 
technique. We will initialize the zerotree with an arbi- 
trary minimal threshold (or accuracy level) and branching 
rate. Notice that the expected distortion associated with 
a given zerotree is a function only of the minimal thresh- 
old employed in that zerotree. Thus the optimal ordering 
for a given threshold and branching rate is the ordering 
that minimizes the expected rate with respect to the data 
set. Roughly speaking, this expected rate is minimized 
by ordering the coefficients from largest to smallest. This 
ordering can be achieved in a number of different ways. 
Most simply, we can order the coefficients according to 
their average energy. Slightly better performance would be 
achieved by ordering the coefficients according to the prob- 
ability with which that coefficient exceeds a given thresh- 
old. 
Given a fixed ordering and minimal threshold, the opti- 
mal branching rate is the branching rate that achieves the 
lowest expected rate. For simplicity we assume a fixed 
branching rate within a given zerotree. Since the number 
of possible branching rates is relatively small, we here con- 
sider all possible branching rates and choose the branching 
rate that minimizes the expected rate with respect to a given 
minimal threshold and ordering. 
Given a fixed ordering and branching rate, the optimal 
minimal threshold is the minimal threshold that results in 
618 
2o r 
18 - 
- - Said Pearlman 
0 002 004 0.06 0.08 0.1 0.12 0.14 
rate 
Figure 1: Comparison of SQNR results on a collection 
of combined text and gray scale images. The systems 
tested include a single optimized-zerotree code and the 
Said-Pearl zerotree code. 
the lowest expected Lagrangian performance for that tree. 
We here consider only thresholds of the form 2k for some 
some integer k less than the logarithm of the maximal 
wavelet coefficient magnitude. As a result, the number of 
thresholds is likewise small, and the optimization of the 
threshold is performed by exhaustive search. 
By iterating the above ordering, branching rate, and 
minimal threshold designs, we achieve a descent algorithm 
on the expected Lagrangian performance that can be iter- 
ated to convergence. 
4 Experimental Results 
In Figure 1, we compare the performance of a single 
optimized zerotree to the performance of a single Said- 
Pearlman zerotree. The optimized zerotree is designed 
using a single 2400 x 2400 image scanned from a page 
of IEEE Spectrum Magazine and tested on another page 
from the same issue. Each page has roughly equal amounts 
of text and gray scale material. The optimized zerotree 
achieves up to 4 dB improvement over the Said-Pearlman 
zerotree. Incorporation of the zerotree design algorithm 
into the optimal weighted universal code design algorithm 
is therefore expected to yield even greater experimental 
performance gains. 
References 
[2] A. Said and W. A. Pearlman. A new, fast, and efficient 
image codec based on set partitioning in hierarchical 
trees. IEEE Transactions on Circuits and Systems for 
Video Technology, 6(3):243-250, June 1996. 
[3] P. A. Chou. Code clustering for weighted universal 
VQ and other applications. In Proceedings of the IEEE 
Intemational Symposium on Information Theory, page 
253, Budapest, Hungary, June 1991. 
[4] P. A. Chou, M. Effros, and R. M. Gray. A vector 
quantization approach to universal noiseless coding 
and quantization. IEEE Transactions on Information 
Theory, IT-42(4):1109-1138, July 1996. 
[5] M. Effros and P. A. Chou. Weighted universal bit allo- 
cation. In Proceedings of the IEEE International Con- 
ference on Acoustics, Speech, and Signal Processing, 
volume 4, pages 2343-2346, Detroit,MI, May 1995. 
IEEE. 
[6] M. Effros and P. A. Chou. Weighted universal trans- 
form coding: universal image compression with the 
Karhunen-Loeve transform. In Proceedings of the 
IEEE International Conference on Image Processing, 
Washington, D.C., October 1995. IEEE. 
[7] M. Effros and P. A. Chou. Universal image compres- 
sion. 1996. Submitted to the IEEE Transactions on 
Image Processing December 18, 1996. 
[l] J. M. Shapiro. Embedded image coding using ze- 
rotrees of wavelet coefficients. IEEE Transactions on 
Signal Processing, 6(3):243-250, December 1996. 
6 19 


Zerotree design for image compression: toward weighted universal zerotree coding

We consider the problem of optimal, datadependent zerotree design for use in weighted universal zerotree codes for image compression. A weighted universal zerotree code (WUZC) is a data compression system that replaces the single, data-independent zerotree of Said and Pearlman with an optimal collection of zerotrees for good image coding performance across a wide variety of possible sources. We briefly describe the weighted universal zerotree encoding and design algorithms but focus primarily on the problem of optimal, data-dependent zerotree design. We demonstrate the performance of the proposed algorithm by comparing, at a variety of target rates, the performance of a Said-Pearlman style code using the standard zerotree to the performance of the same code using a zerotree designed with our algorithm. The comparison is made without entropy coding. The proposed zerotree design algorithm achieves, on a collection of combined text and gray-scale images, up to 4 dB performance improvement..

Michelle Effros

CiteSeerX

Zerotree Design for Image Compression: Toward Weighted Universal Zerotree Coding

A new, fast, and efficient image codec based on set partitioning in hierarchical trees.

A vector quantization approach to universal noiseless coding and quantization.

Code clustering for weighted universal VQ and other applications.

Embedded image coding using zerotrees of wavelet coefficients.

Universal image compression.

Weighted universal bit allocation.

Weighted universal transform coding: universal image compression with the Karhunen-Loeve transform.

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.56.177

Zerotree design for image compression: toward weighted universal zerotree coding

Abstract

Similar works

Full text

Available Versions

Caltech Authors - Main

CiteSeerX