This paper introduces a new HDTV coder based on motion compensation, subband coding, and high order conditional entropy coding. The proposed coder exploits the temporal and spatial statistical dependencies inherent in the HDTV signal by using intra- and inter-subband conditioning for coding both the motion coordinates and the residual signal. The new framework provides an easy way to control the system complexity and performance, and inherently supports multiresolution transmission. Experimental results show that the coder outperforms MPEG-2, while still maintaining relatively low complexity

Chung, Wilson

Kossentini, Faouzi

Smith, Mark J. T.

English

NASA Technical Reports Server

NASA-CR-20_398
F ¢,'l
A SUBBAND CODING METHOD FOR HDTV
Wilson Chung, Faouzi Kossentini, and Mark J. T. Smith
School of Electrical & Computer Engineering
Georgia Institute of Technology
Atlanta,GA 30332
This paper introduces a new HDTV coder based
on motion compensation, subband codling, and high
order conditional entropy coding. The proposed coder
exploits the temporal and spatial statistical dependen-
cies inherent in the HDTV signal by using intra- and
inter-subband conditioning for coding both the motion
coordinates and the residual signal. The new frame-
work provides an easy way to control the system com-
plexity and performance, and inherently supports mul-
tiresolution transmission. Experimental results show
that the coder outperforms MPEG-2, while still main-
taining relatively low complexity.
I. INTRODUCTION
Several methods have been proposed recently for trans-
mission of HDTV [1, 2, 3, 4, 5, 6, 7]. Most employ mo-
tion compensation at one stage or another, after which the
residual between the original and predicted frames is com-
puted and encoded spatially. DCT-based spatial coders
are widely used, most notably in the MPEG standards.
However, subband coders are also becoming popular.
There are many important issues that are associated
with HDTV coding, such as control over the bit rate and
picture quality, error correction and concealment, and mul-
tiresolution capability for multisource decoding and pro-
gressive transmission applications. In this paper, we intro-
duce a new subband video coder which achieves good per-
formance with low relative complexity, but also provides
a framework where most of these issues can be easily ad-
dressed. The proposed coder employs motion estimation
and compensation independently for each subband, but
encodes the motion vectors using a high order conditional
entropy coding scheme that exploits statistical dependen-
cies between motion vectors of the same frame and suc-
cessive frames as well as between the coordinates of the
motion vectors, simultaneously. The coder also identifies
non-compensatable blocks through the use of statistically
optimized thresholding, which are then intra-frame coded.
The video coder is described next. This is followed by a
discussion of practical design issues. Section 4 presents
Thiswork was supportedby theNationalScienceFoundation
under contract MIP-9116113 and the National Aeronautics and
Space Administration.
experimental results which compare the performance and
complexity of the coder with that of MPEG-2.
II. THE VIDEO CODER
First,considera conventionalsubband video coder. In
the parlanceofMPEG, the frames that arecoded spatially
are calledIframes. Those that are forward-predictedare
calledP frames,and thosethat areforward- and backward-
predicted are calledB frames. The sequence of video
frames isfirstgrouped into blocks of N frames, where
the firstframe (or I frame) iscoded using an intra-frame
subband coder,and the other N - I frames (or P frames)
are predictedusing motion estimation and compensation,
and the residualframes are coded using another subband
coder. Inthiswork, no B frames are used. At the receiver,
each video frame isconstructedfrom motion information
(ifapplicable)and the coded residualframe.
There are two important problems associatedwith the
above coder. First,motion compensation using the block
matching algorithm with a typicalblock sizeof 16 x 16
and search range of-16 -to-+16 in each dimension is
usuallycomputationally intensive.This problem becomes
even worse in IIDTV coding because both block sizesand
search areashave to be somewhat largerto achieve good
performance. Second, due to the block matching algo-
rithm,blockinessfrequentlyappears inthe residualframe,
which introdacesartificialhigh frequencies.To solvethese
two problems, we apply the block matching algorithm to
each ofthe subbands. Figure 1 shows a block diagram of
the proposed subband coder and Figure 2 shows the struc-
ture of the RVQ coder. Each frame isfirstdecomposed
into subbands using a tree-structuredfIR analysisfilter
bank. The filterbank .isbased on two-band decomposi-
tions,which employ allpasspolyphase separableIIR filters
[8].A full-searchblock matching algorithm (BMA) using
the mean absolutedistance(MAD) isused to estimatethe
motion vectors.Since the BMA does not necessarilypro-
duce the true motion vectors,we employ a thresholding
technique forimproving the rate-distortionperformance.
Let d,ninbe the minimum MAD associatedwith a block
to be coded. Also, letT be a threshold,which isa large
positivenumber empiricallydetermined from the statis-
https://ntrs.nasa.gov/search.jsp?R=19970017961 2020-06-16T01:54:34+00:00Z
HWrV
YUV
vk_.o
Xi
.-----a
Dec.ore- i ,
V 1.RvQc_,I
L
_,_ RVQ Coder _.
_oad_6oa£ l
E.,o_r _ I Dit_
RVQ_ i
J
i
I
Hi_(_ler .
r_y
_' ,_
--I
Figure 1: Proposed subband video coder.
C'm_mt MC-Subbandi
t '
ME an __] s_
1MC-P
7
w
RVQ symbols
I
Motion Vectors
l
Figure 2: Basic structure of the RVQ coder.
tics of the subband being coded. If drn_n > T, then the
block is likely not compensatable. Thus, both the origi-
nal block and the residual block, obtained by subtracting
the motion compensated predicted block from the original
one, are coded using the intra-band and residual coders,
respectively, and the One leading to better rate-distortion
performance is chosen (as will be described shortly). A
special symbol, which can be coded as part of the motion
information, is sent to the decoder indicating the type of
coding used.
In many conventional RDTV subband coders as well
as in MPEG, differential entropy coding of motion vec-
tors is employed. Since motion vectors are usually slowly
varying, the motion bit rate can be further reduced by
exploiting dependencies not only between previous mo-
tion vectors within and across the subbands but also be-
tween the vector coordinates. For this purpose, we em-
ploy a high order conditional entropy coder that is based
on finite state machine (FSM) modeling. More specif-
ically, let (X.,,n, Ymm) be the pair of random variables
representing the current horizontal and vertical motion
displacements in the current subband (n, m) in frame n.
Also, let (Umm, Vmm) be the pair of state random vari-
ables with realizations an,r. = {0, 1,..., S_n,m}and vn,m =
{0, 1,...,S_.,m}, which we associate with Xmm and Yn.,.,
respectively. Each state un,m is given by
0 1 8M,,,,,_'_
u.,,_ = Y.,,_(sn,,_,sn,m, .'., .,,n I,
and each state v.,m is given by
v.,,_ = G.,._(_.,,n,tn,rn, ...,-.,m ,,
o 1 S-M_" and ,o -,_ ._,. • .,t_h = are
where an,m,an,m,•••, ,,,,,. ,.,.....,..- .
previously coded conditioning symbols. The mappings
F.,,n and Gn,m are generally many-to-one mappings that
convert combinations of realizations of the conditioning
symbols to a particular state. Assuming that X.,m is en-
tropy coded first, the conditioning symbols for the FSM
model associated with Xn,m are selected from a region
composed of symbols located in all previously coded sub-
bands (i.e., where motion vectors were already coded) in
both frames n and n - 1. When Yn,m is being coded, the "
horizontal displacements in the same subband can also be
included in the conditioning region.
Statistical modeling for entropy coding the motion vec-
tors consists of first selecting, for each subband (n, m),
Mmm (Nn,r.) conditioning symbols for X.,m (Yn,m), and
then finding mappings Fn,m and Gn,,n such that the con-
ditional entropies H (X,_,,_ IUn,m ) and H (X.,m lU.,m ) are
minimized subject to a limit on complexity. The total
number of probabilities that must be computed and stored
is used here as a measure of complexity. The tree-based al-
gorithms described in [9] are used to find the best values of
Mn,m and N.,m subject to a limit C; on the total number
of probabilities. The PNN algorithm [10], in conjunction
with the generalized BFOS algorithm [11], is then used to
construct mapping tables that represent Fn,m and G,,m
subject to another limit C2 (C2 << C1) on the number of
probabilities.
The intra-band (I-subband) and residual (P-subband)
coders are multistage residual vectors quantizers (RVQs)
followed with high order conditional statistical models,
which are optimized to the intra-band and residual band
statistics, respectively. Multistage RVQs provide an easy
way to control the complexity-performance tradeoffs, and
allow efficient high order statistical modeling. We restrict
the number of code vectors per stage to be 2, which sim-
plifies both statistical modeling and entropy coding used
in this work. This also provides the highest resolution in
a progressive transmission environment.
The same statistical modeling algorithm used for en-
tropy coding the motion vectors is also used for entropy
coding of the output of the' RVQs. Both the motion vec-
tors and the output of the RVQs are eventually coded
using adaptive binary arithmetic coders (BACs) [12, 13].
These coders are very easy to adapt and require small
complexity.
HI. PRACTICAL DESIGN ISSUES
To achieve the lowest bit rate, the statistical models
used to entropy code the motion vectors should be gen-
erated on-line. However, this requires a two-pass process
where statistics are generated in the first pass, and the
statistical modeling algorithm described above is used to
generate the conditional probabilities. These probabilities
must then be sent to the BAC decoders so that they can
track the corresponding encoders. In most cases, this re-
quires a large complexity. Moreover, even by restricting
the number of states to be relatively small (such as 8), the
side information can be excessive. Therefore, we choose
to initialize the encoder with a generic statistical model,
which we generate using a training HDTV sequence, and
then employ dynamic adaptation [12] to track the local
statistics of the motion flow.
For both the I-subbands and P-subbands, the multi-
stage RVQs and associated statistical models are designed
jointly using an entropy and complexity-constrained algo-
rithm, which is described in [9, 14]. The design algorithm
iteratively minimizes the expected distortion E{d(X, X))
subject to a constraint on the overall entropy of the sta-
tistical models. The algorithm is based on a Lagrangian
minimization and employs a Lagrangian parameter A to
control the rate-distortion tradeoffs. To substantially re-
duce the complexity of the design algorithm, only sepa-
rate subband encoders and decoders are used. However,
the RVQ stage encoders in each subband are jointly opti-
mized through dynamic M-search, the decoders are jointly
optimized using the Gauss-Seidel algorithm.
The most important part of the design algorithm is
the encoding procedure, where either an intra-frame or
inter-frame subband coder must be chosen for a particu-
Bn,,_ of sizelar block. Suppose we want to encode a block
Ln,m using the proposed I-subband and P-subband coders
with Lagrangian parameters (or quality factors) A1 and
Ap, respectively. The BMA algorithm is first applied, and
the minimum MAD d,ni, is computed. If dmi, < T, then
the corresponding motion vector is encoded using the BAC
specified by the current state, and the residual block is
quantized using the P-subband (residual) RVQ. The out-
put of each RVQ stage is encoded with a separate entropy
coder composed of a FSM statistical model and a set of
BACs, each specified uniquely by a state. If dmi, >_ T,
then the block is both I-subband and P-subband coded.
Let P_ = -log2p(zi[u i) and P_ = "log2 p(yi[v _) be esti-
mates of the number of bits required to code the horizontal
and vertical coordinates of the motion vector, respectively.
Also, let dp be the distortion and Rp be the rate that
compose the minimum Lagrangian Je =dp -4-ApRp as-
sociated with coding the residual block. Assuming that
J1 = dl -4- AIR1 is the minimum Lagrangian associated
with coding the original block, then the l-subband coding
Figure 3: The ll4th frame of the sequence Bp.rrs
method is selected if
J1 < dp + _j,(/_ +/_ + Re).
The proposed coder has many practical advantages,
due to both the subband structure and the multistage
structure of RVQ. For example, multiresolution transmis-
sion can be easily implemented in such a framework. An-
other example is error correction, where the more probable
of the two stage code vectors is selected if an uncorrectable
error is detected. Since each stage code vector represents
only a small part of the coded vector, this should not sig-
nificantly affect the reconstruction or the FSM statistical
models.
IV. EXPERIMENTAL RESULTS
The image shown in Figure 3 isframe number 114 of
the testsequence BRITS, which we encode using both the
proposed coder and MPEG-2. The frame sizeis720 x 1280.
The originalRGB colorsequence with 8 bits/pixelrequires
approx 1.3 Gbs. The MPEG-2 software we used resides
on flp.ne_com.com:lpublcfogg/mpeg$ [15].
In our experiments, each frame is decomposed into 64
uniform subbands, but more than half of the subbands
are not coded. This is determined based on initial rate-
distortion tradeotIs [9]. The BMA algorithm used in our
experiments employs a block size of 2 x 2 and a search
area of -2 -to- -t-2 in each dimension. Motion estima-
tion is performed, and is done only for the Y luminance
component and the estimated motion vector field is sub-
_-quently used for the motion compensation of U and V
¢hrominance signals. A high order conditional entropy
coder is designed for the motion vector coordinates, and
one I-subband coder and one P-subband coder with vector
size of 2 × 2 are designed for the each of the YUV com-
ponents. We set the maximum allowed numbers of condi-
tional probabilities for the motion entropy coder and the
I-subband and P-subband entropy coders to C1 - 4094
and C2 - 512. The BAGs used employ a skew factor
between 1 and 256.
41
0+I
0.I
O.7
O.l
|oJ
_2
0.1
(a)
J
j-
£
\
i IItPEG-2
It
+ _, ,_ ,_ +, + +; + + ,',,
Flmme W
(b)
m • •
++
s
+
+
+
+
+
+
k, fms_ _
%
•" • MPEG-_
+ ++ + _ +, + , a . ,o
I: PI_I l_rnb4l+
Figure 4: (a) Overall rate usage and (b) PSNR performance for the
proposed coder.
For each rate-distortion point, the total memory re-
quired to store both the I-subband and P-subband RVQ
codebooks and associated tables of conditional entropy
codes is approximately 4.6 kilobytes. Moreover, only 512
bytes are required by the motion entropy coder. For anal-
ysis, quantization using dynamic M-search, and BAC en-
coding, approximately 27 multiplies and 32 adds per pixel
are required. Only 3 multiplies and 14 adds are required
for BAC decoding, inverse quantization, and synthesis.
Not only are the encoding complexity and memory rela-
tively small, but the performance is also good. Figure 4 (a)
shows the average bit per pixel and Figure 4 (b) shows the
PSNR result of our coder in comparison with the MPEG-
2 standard for 10 frames of the luminance component of
the color test video sequence BAITS. The average bit rate
is approximately 18.0 Mbits/seC and the average PSNR
is 34.75 dB for the proposed subband coder and 33.70
dB for MPEG-2. As is shown in the figure, the proposed
coder clearly outperforms MPEG-2. Moreover, although
MPEG-2 requires less encoding complexity and memory,
the complexity of our subband coder are still reasonable.
ltEFERENCES
[1] J. G. Apostolopoulos _nd J. S. Lira, "Video compression for
digital a_tvanced television systems," in Motion Analysis ond
Image Sr4_ence Processing, Bc,ston: Kluwer Academic Pub-
fishers, 1992.
[2] S. M. Lei, T. C. Chen, and K. H. Tzou, "Subband HDTV
coding using high-order conditional statistics," IEEE Journal
on Selecled AreaJ i_ Communications, vol. 11, pp. 65-76, Jan.
1993.
[3] J. Belllsio and K. T_ou, "HDTV find the merging broudband
isdn network," SPIE Pros. Visual Comm_nicationa and lm_ge
processing, vol. 1001, pp. 772-786, 1988.
[4] F. Bosveld, 11. La_endijk, and J. Biemond, "Hierarchical coding
of HDTV," Sig_ol Processing: Imoge Comm,nication, vol. 4,
pp. 195-225, July 1992.
[5] F. Bosveld, 11. Lagendijk, and J. Biemond, "Compatible
HDTV transmiuionusing conditional entropy coding," in Proc.
ICASSP, vol. V, (Minneapolis, MN, USA), Apr. 1993.
[6] I. Furulmwa, M. Nomura, N. Oth_, and S. Ono, "Hierarchica/
coding of super high definition hn_ges with adaptive block-size
multi-stage VQ," S@nol Processing of HDTV, vo]. III,1992.
[7] S. Wu and A. Gersho, "]_te-constralned optimal block-
adaptive coding for digital t_pe recording of HDTV," IEEE
Trans. on CVST vol. 1, pp. 100-112, March 1991.
[8] M. Smith and S. Eddins, "An_ysis/synthesis techniques for
subband image coding," IEEE Trans. on ASSP., vol. 38,
pp. 1446-1456, Aug. 1991.
[9] F. Kossentini, W. Chunk, and M. Smith, "A jointly optimized
subband coder," S_bmit_ed io Transactions on IP., July 1994.
[10] W. H. F.,quitz, "A new vector qu_ntization clustering algo-
rithm," IEEE Trans. ASSP., pp. 1568-1575, October 1989.
[II] E. A. Riskin, "Optima] hit allocation via the generated BFOS
algorithm," IEEE Trans. on IT, vol. 37, pp. 400-472, Mar.
1991.
[12] G. G. Lan_don and J. Rissanen, "Compression of black-white
images with arithmetic coding," IEEE Tra,s. o_ COM, vol. 29,
no. 6, pp. 658-867, 1981.
[13] W. B. Pennebaker, J. L. Mitchell, G. G. L_ngdon, and 11. B.
Arps, "An overview of the basic principles of the Q-coder
adaptive binary arithmetic coder," IBM J. Rcs. Def., vol. 32,
pp. 717-726, Nov. 1988.
[14] F. Kossentird, W. Chung, and M. Smith, "Subbandimage cod-
in_ with jointly optimized qum_tizers," Abstract S_bmitted to
ICASSP, (Detroit, MI, USA), Apr. 1995.
[15] MPEG Software Simulation Group, "MPEG-2 encoder / de-
coder, version 1.0, _ ISO/IEC DIS 15818-$ ¢odec, May 1995.


A Subband Coding Method for HDTV

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server