It is known that the distribution of the discrete cosine transform (DCT) coefficients of most natural images follow a Laplacian distribution, and this knowledge has been employed to improve decoder design. However, such is not the case for text documents. In this letter, we present an analysis of their DCT coefficient distributions, and show that a Gaussian distribution can be a realistic model. Furthermore, we can use a generalized Gaussian model to incorporate the Laplacian distribution found for natural images.published_or_final_versio

Lam, EY

IEEE Signal Processing Letters

HKU Scholars Hub

Title Analysis of the DCT coefficient distributions for documentcodingAuthor(s) Lam, EYCitation Ieee Signal Processing Letters, 2004, v. 11 n. 2 PART I, p. 97-100Issued Date 2004URL http://hdl.handle.net/10722/42950Rights Creative Commons: Attribution 3.0 Hong Kong LicenseIEEE SIGNAL PROCESSING LETTERS, VOL. 11, NO. 2, FEBRUARY 2004 97Analysis of the DCT Coefficient Distributionsfor Document CodingEdmund Y. Lam, Member, IEEEAbstract—It is known that the distribution of the discrete co-sine transform (DCT) coefficients of most natural images followa Laplacian distribution, and this knowledge has been employedto improve decoder design. However, such is not the case for textdocuments. In this letter, we present an analysis of their DCT coef-ficient distributions, and show that a Gaussian distribution can bea realistic model. Furthermore, we can use a generalized Gaussianmodel to incorporate the Laplacian distribution found for naturalimages.Index Terms—Discrete cosine transform (DCT), document pro-cessing, image analysis, image coding, probability statistics.I. INTRODUCTION AND MOTIVATIONE LECTRONIC document processing is getting more im-portant these days with the widespread practice of “pa-perless” office. In particular, to save on the memory requiredto archive the documents, a good compression mechanism isneeded. Specialized compression schemes have been developedfor texts, such as JBIG and JBIG2 [1]. However, there is also atendency to use methods designed for natural images in docu-ment compression as well, such as when the application does notsupport these text compression techniques. This is also neededfor mixed document compression, where it may contain text,line art, images, and background together [2].Text and natural images have many different characteristics,and compression schemes such as JPEG [3] and JPEG 2000[4] that are designed for the latter may not work well for theformer. For example, it is known that using JPEG on text, evenwith moderate compression ratio, there are noticeable artifactsalong the edges that degrade the visual quality. In this letter, wefocus our attention on analyzing the distribution of the discretecosine transform (DCT) coefficients for text documents. It hasbeen justified that such distribution resembles a Laplacian dis-tribution for natural images [5], and this knowledge has beenused to improve the decoding in JPEG [6], [7]. Fig. 1(a) showsa typical grayscale image, and Fig. 1(b) shows the distribution ofone of the DCT coefficients. The dotted line represents the max-imum-likelihood Laplacian fit of the empirical data. As seen inthe Figure, the data match the distribution rather well. On theother hand, consider Fig. 2. Fig. 2(a) shows a typical text image,Manuscript received November 8, 2002; revised February 7, 2003. This workwas supported by the University Research Committee in the University of HongKong under Grant Number URC-10204526. The associate editor coordinatingthe review of this manuscript and approving it for publication was Prof. Scott.T. Acton.The author is with the Department of Electrical and Electronic Engineering,University of Hong Kong, Hong Kong (e-mail: elam@eee.hku.hk).Digital Object Identifier 10.1109/LSP.2003.821789(a)(b)Fig. 1. Grayscale image with one DCT coefficient distribution.and Fig. 2(b) shows the distribution of the corresponding DCTcoefficient. Again we show the maximum-likelihood Laplacianfit of the empirical data. This is not a good fit. Therefore, if wedesign a decoder that shifts the decoding value because the co-efficient behaves like a Laplacian distribution [6], it may not be1070-9908/04$20.00 © 2004 IEEE98 IEEE SIGNAL PROCESSING LETTERS, VOL. 11, NO. 2, FEBRUARY 2004(a)(b)Fig. 2. Text image with one DCT coefficient distribution.applicable for text images. In Section II, we formulate a math-ematical framework to examine what would be a better modelfor the DCT coefficient distributions for text.II. DCT COEFFICIENT DISTRIBUTION MODELINGIn [5], it was shown that a doubly stochastic model is veryhelpful to provide us an insight into the distribution. In thismodel, we argue that within an 8 8 block used for DCT, as-suming that the pixels are identically distributed, the DCT coef-ficient is approximately Gaussian. Let be the coefficient, andbe the variance of the block, we have(1)Fig. 3. Distribution of block variance for a text image.However, the block variance is itself a stochastic quantity. Theactual DCT coefficient distribution is given by(2)It has also been shown that for natural images, the distributionof the block variance is close to exponential, in which case wecan put in and (1) in (2) to get(3)which largely explains why the empirical distribution for DCTcoefficients for natural images are close to Laplacian [5]. On theother hand, this is not the case for text documents. Fig. 3 showsthe distribution of the block variance for Fig. 2(a). As seen inthe diagram, there are two components in the distribution.1) A large concentration of the variance at or near zero.2) A nearly uniform distribution of the variance otherwise.The first component corresponds to a flat region, which repre-sents the background in the document. We focus on the con-tribution from the second component, because only the lowestDCT coefficient will be affected if a region has zero variance,and this coefficient is not modeled with any known distributionanyway. We will analyze the case where the block variance isuniform, and the case that it is Gamma-distributed. The secondcase will help us tie into the exponential distribution for naturalimages.A. Uniform Distribution of Block VarianceWe put for in (2), and obtain(4)when we substitute for . We cannot find a closed-form solu-tion for (4), but we can perform the integration numerically, andLAM: ANALYSIS OF THE DCT COEFFICIENT DISTRIBUTIONS 99Fig. 4. DCT coefficient distribution with uniform block variance.the result is plotted in Fig. 4. We observe that its shape resem-bles a normal distribution, so we also plot the closest Gaussiancurve in the figure.B. Gamma Distribution of Block VarianceGamma distribution is defined as(5)where is the gamma function. is called the shape param-eter, while is called the scale parameter [8]. The Gamma distri-bution becomes the exponential distribution for . As in-creases, the mean of the distribution will also shift up. Althoughthe Gamma distribution is more complicated than the uniformdistribution, when put it in (2) there is actually a closed-formsolution(6)for integer value of . The detailed derivation can be found in[9]. Fig. 5 plots (6) for . Again we also plot the closestGaussian curve in the figure.Since is a linear function of , we can infer fromthe above two special cases that a block variance distributionclose to a linear combination of them will lead to a DCT coef-ficient distribution resembling Gaussian. The data as plotted inFig. 3 is seen to be one such case. As a verification to the aboveanalyses, we use the data in Fig. 2(b) but ignore the coefficientsnear zero, which would mainly come from the background of thetext. We plot the maximum-likelihood estimate of the Gaussiandistribution to the data in Fig. 6. It confirms our analysis that aGaussian fit to the coefficient is better for text document.Note that the Laplacian and Gaussian distributions can beconsidered special cases for the generalized Gaussian distribu-tion. Its probability density function, with zero mean, is(7)Fig. 5. DCT coefficient distribution with gamma block variance.Fig. 6. Gaussian fit to empirical coefficient distribution of text document.where controls the shape of the distribution and thespread. When and , it becomes a standardGaussian distribution. When and , it becomesa Laplacian distribution with parameter . Text and natural im-ages therefore can be considered to produce coefficient distri-butions with generalized Gaussian distribution having differentshape parameters. Combining this study with the study of DCTcoefficient distributions for natural images, we argue that thegeneralized Gaussian distribution is a very versatile model forsuch a purpose. The shape parameter is around 1 for naturalimages, and near 2 for text. This will be especially helpful formixed document compression, where both images and text arepresent [10]. We compute the shape parameters for the image inFig. 2(a), and plot the results for the 63 ac coefficients in Fig. 7.In performing this fitting, we use the method described in [11],and ignore the blocks with near zero variance. We can see thatusing is a good model for text documents.100 IEEE SIGNAL PROCESSING LETTERS, VOL. 11, NO. 2, FEBRUARY 2004Fig. 7. AC coefficients for generalized Gaussian fit to text document.III. CONCLUSIONIt has been shown that for natural images, the coefficient dis-tribution resembles a Laplacian distribution. This letter extendsthe study into text documents. Although the model would callfor rather complicated equations such as (4) and (6) to computethe distribution, we find that a Gaussian distribution is a muchsimpler model that is still rather accurate. This can further beconsidered a special case of the generalized Gaussian distribu-tion, and can combine with the Laplacian distribution for naturalimages under the same model. Since the distribution of the co-efficients has been widely studied and used in decoder design,the study presented here can serve as an aid to improve text doc-ument compression, while the actual coding gain still needs tobe explored.ACKNOWLEDGMENTThe author wishes to thank P. Y. Yeung for the document inFig. 2(a) and the University Research Committee in the Univer-sity of Hong Kong for financial support.REFERENCES[1] I. Witten, A. Moffat, and T. Bell, Managing Gigabytes: Compressingand Indexing Documents and Images, 2nd ed. New York: MorganKaufmann, 1999.[2] J Li and R. Gray, “Context-based multiscale classification of documentimages using wavelet coefficient distributions,” IEEE Trans. Image Pro-cessing, vol. 9, pp. 1604–1616, Sept. 2000.[3] W. Pennebaker and J. Mitchell, JPEG Still Image Data CompressionStandard. New York: Van Nostrand Reinhold, 1992.[4] D. Taubman and M. Marcellin, JPEG 2000: Image Compression Fun-damentals, Standards and Practice. Boston, MA: Kluwer, 2001.[5] E. Y. Lam and J. W. Goodman, “A mathematical analysis of the DCT co-efficient distributions for images,” IEEE Trans. Image Processing, vol.9, pp. 1661–1666, Oct. 2000.[6] G. Lakhani, “Distribution-based restoration of DCT coefficients,” IEEETrans. Circuit Syst. Video Technol., vol. 10, pp. 819–823, Aug. 2000.[7] J. Price and M. Rabbani, “Biased reconstruction for JPEG decoding,”IEEE Signal Processing Lett., vol. 6, pp. 297–299, Dec. 1999.[8] J. Rice, Mathematical Statistics and Data Analysis, 2nd ed. Belmont,CA: Duxbury, 1995.[9] D. Teichroew, “The mixture of normal distributions with different vari-ances,” Ann. Math. Stat., vol. 28, no. 2, pp. 510–512, June 1957.[10] E. Y. Lam, “Analysis of the DCT coefficient distributions for documentcoding,” in Proc. Digital Photography Conf.: PICS 2003, May 2003, pp.518–521.[11] K. Sharifi and A. Leon-Garcia, “Estimation of shape parameter for gen-eralized Gaussian distributions in subband decompositions of video,”IEEE Trans. Circuit Syst. Video Technol., vol. 5, pp. 52–56, Feb. 1995.

Analysis of the DCT coefficient distributions for document coding

http://hub.hku.hk/bitstream/10722/42950/1/88828.pdf

Analysis of the DCT coefficient distributions for document coding

Abstract

Similar works

Full text

Available Versions

HKU Scholars Hub