This paper investigates the relation between error distribution and predictive order of minimum mean abusolute error predictors (MMAE predictors) designed for lossless coding of grayscale images. Design of MMAE predictors reduces to the linear programming problem. Let k be the number of coefficients in a predictor (predictor order), we imagine that predictor order k may have a distribution shaping effect. Main purpose of this paper is to ensure that k has such an effect

Morikawa, Yoshitaka

Ueno, Hirokazu

Okayama University Scientific Achievement Repository

Hirokazu UENO¤Graduate School of Natural Science andTechnology, Okayama University3-1-1, Tsushima-naka, Okayama, JapanYoshitaka MORIKAWAyGraduate School of Natural Science andTechnology, Okayama University3-1-1, Tsushima-naka, Okayama, JapanThis paper investigates the relation between error distribution and predictive order of minimum meanabusolute error predictors(MMAE predictors) designed for lossless coding of grayscale images. Design ofMMAE predictors reduces to the linear programming problem. Let k be the number of coefficients in apredictor(predictor order), we imagine that predictor order k may have a distribution shaping effect. Mainpurpose of this paper is to ensure that k has such an effect.1 INTRODUCTIONRecent years have seen an increased level of research inlossless image compression, in addition to lossy compres-sion. Lossless image codings are required and desired incertain applications such as medical and satellite imagingsand digital archiving of cultural heritages. Since the predic-tive coding scheme enables us to predict each pixel one byone and rather precisely in aid of adaptation, many losslesscoding schemes employ prediction.For lossless image coding based on prediction, the cod-ing performance depends largely on the efficiency of pre-dictors. Many lossless image coding formats use minimummean square error predictors(MMSE predictors)[1][2][3], butMMSE predictors are susceptible to edges(the big transitionpart of the brightness value) in images. So Hashidume etal.[4] have proposed minimum mean absolute error predic-tors(MMAE predictors)[4] which are robust to edges. [4]says that using MMAE predictors the accuracy of predictionis enhanced and entropy of prediction is reduced.This paper investigates the relation between error distri-bution and predictor order of MMAE predictors designed forlossless image coding. Design of MMAE predictors reducesto the linear programming problem. Let k be the number ofcoefficients in a predictor(predictor order), at least k predic-tion errors become 0. Thus, we imagine that predictor orderk may have a distribution shaping effect. Main purpose ofthis paper is to ensure that k has such an effect.¤ueno@trans.cne.okayama-u.ac.jpymorikawa@cne.okayama-u.ac.jpIn this paper, we define the fitting degree of actual predic-tion error distribution and the modeling distribution(Laplacedistribution) in terms of the redundancy(the difference of realentropy and model entropy), and measure the redundancychanging the value of k. Then we found that the optimumk appears to minimize the redundancy.2 MINIMUM MEAN ABSOLUTE ERRORPREDICTORWhen we denote the current pixel pi’s value B(pi), thepredicted value B^(pi) is calculated byB^(bi) = µTi ¢ a; (1)where µi = [B(pi1); B(pi2); ¢ ¢ ¢ ; B(pik)]T is the local cau-sal area(support region) vector of pi (see Fig. 1 in the nextpage) and a = [a1; a2; ¢ ¢ ¢ ; ak]T is the vector of predictioncoefficients for pi. When we denote the set of pixels in cod-ing area R = fpiji = 1; 2; ¢ ¢ ¢ ; Sg, a problem to designa MMAE predictor for R can be written as a mathematicalprogramming problem as follows:Minimizeak e k1 =Xpi2Rjeijsubject to e = B ¡ B^;e = [e1; e2; ¢ ¢ ¢ ; eS ]T ;B = [B(p1); B(p2); ¢ ¢ ¢ ; B(pS)]T ;B^ = [B^(p1); B^(p2); ¢ ¢ ¢ ; B^(pS)]T ;B^(pi) = µTi ¢ a (for i = 1; 2; ¢ ¢ ¢ ; S);a = [a1; a2; ¢ ¢ ¢ ; ak]T :(2)This work is subjected to copyright.All rights are reserved by this author/authors.Predictor Order and Error Distribution of MMAE Predictors forLossless Image Coding(Received November 28, 2008)Memoirs of the Faculty of Engineering, Okayama University, Vol. 43, pp. 93-98, January 2009932x9ip7ip11ip12ip14ip16ip19ip20ip13ip17ip15ip18ip8ip10ip1ip2ip3ip4ip5ip6ipipFig. 1: Pixels of support region(k = 20). The position ofpixels in support region is defined by the Manhattan distancefrom the current pixel.The absolute part of the objective function makes thisproblem difficult to solve. So, we transcribe the ith elementof the vector e asei = e+i ¡ e¡i ; e+i >= 0; e¡i >= 0; (3)and boil down this problem to a linear programming prob-lem. In the same way, transcribing aj which is the jth ele-ment of the prediction coefficients a, the problem (2) can berewritten as a linear programming problem as follows:Minimizea1T ¢ e+ + 1T ¢ e¡subject to B^ + e+ ¡ e¡ = B;e+ = [e+1 ; e+2 ; ¢ ¢ ¢ ; e+S ]T >= 0;e¡ = [e¡1 ; e¡2 ; ¢ ¢ ¢ ; e¡S ]T >= 0;B = [B(p1); B(p2); ¢ ¢ ¢ ; B(pS)]T ;B^ = [B^(p1); B^(p2); ¢ ¢ ¢ ; B^(pS)]T ;B^(pi) = µTi ¢ (a+ ¡ a¡)(for i = 1; 2; ¢ ¢ ¢ ; S);a+ = [a+1 ; a+2 ; ¢ ¢ ¢ ; a+k ]T >= 0;a¡ = [a¡1 ; a¡2 ; ¢ ¢ ¢ ; a¡k ]T >= 0;1 = [1; 1; ¢ ¢ ¢ ; 1]T :(4)This problem could be solved by a linear programmingmethod such as the simplex method or the interior-point met-hod. In this paper, we employ Barrodale’s method[5] whichis based on the simplex method. Also, our lossless codingscheme employs the classfication-based technique[3]: eachdivided block(8 £ 8 pixels) of a image is classified to selectan appropriate linear predictor based on a MMAE criterionfrom C different kinds of predictors, and each predictor isoptimized for each class of blocks.Table 1 lists the entropy of whole prediction errors foreach the standard image[4]. The definition of the entropy isshown in section 4. From Table 1, we can see that all of theEntropy of MMAE is smaller than MMSE. Thus, the predi-ciotn accuracy of MMAE predictors is better than MMSEpredictors.3 ERROR DISTRIBUTIONWe encode the error image (the difference of the orig-inal image and prediction image) using the entropy codingfor lossless image coding. Thus, we need event probabilitiesTable 1: Entropy(bits/pixel) of prediction errors only. Thistable indicates that the prediction accuracy of MMAE pre-dictors is better than MMSE predictors.Image MMSE MMAEairplane 3.675 3.618baboon 5.731 5.693balloon 2.714 2.643barb 4.123 3.969barb2 4.398 4.302camera 4.048 3.967couple 3.428 3.386goldhill 4.304 4.242lena 4.353 4.295lennagrey 3.968 3.913peppers 4.277 4.224Average 4.092 4.023of prediction errors. But if we encode it using actual eventprobabilities, the additional information will be increased,because when we decode, we need to use the same eventprobabilities as those of encoding. On the other hand, if wemodel actual event probabilities, and encode errors using themodel event probabilities, the additional information will benegligible, because it is only the scale parameter to deter-mine model event probabilities. So, it is important to knowthe shape of the error distribution.3.1 Modeling of The Error DistributionIn general, the prediction error distribution is a mixturedistribution of some different scale parameters. In order toeffectively tackle this problem, we divide the prediction er-rors into some groups by modeling. For classified groups, wemay obtain the high coding performance by assigning differ-ent encoders.In this papers, we use the modeling method called con-text modeling.3.2 Context ModelingA parameter Ui of a current pixel pi is defined as follows:Ui =6Xj=11dijjeij j; (5)where the pixel pij is the pixel in the local causal area given1ip2ip3ip4ip5ip6ipipFig. 2: Region of pixels for the context modeling.Hirokazu UENO and Yoshitaka MORIKAWA MEM.FAC.ENG.OKA.UNI. Vol. 4394in Fig. 2, and dij is the Manhattan-distance between it andthe current pixel pi.Because the correlation between parameter Ui and vari-ance ¾2 of the prediction error of the current pixel pi is verystrong[4], we divide into some groups for the prediction errorusing a parameter U in cotext modeling.We use values called threshold when we divide into somegroups. So, we prepare thresholds as follows:Th0 <= Th1 <= ¢ ¢ ¢ <= ThN¡1;where N is the number of groups. Specifically we clas-sify the prediction errors into N groups by the parameter U .Futhermore, we optimize those thresholds in order to mini-mize the information value of pixels. So, we need to estimatethe information value of the prediction error. The informa-tion value of the prediction error of a current pixel is definedas follows.Since the pixel values and the predictive values of graysc-ale are in the range between 0 and 255, the possible valuesof a prediction error are bounded between ¡255 and 255.Therefore, when the predictive value B^(pi) of the currentpixel pi and it’s quantization group gi are given, the condi-tional probability of the prediction error ei is defined asPrgi(eijB^(pi); gi) =Prgi(eijgi)P255x=¡255 Prgi(xjgi); (6)where Prgi(xjgi) is given by Probability Density Function(PDF) f(x) asPrgi(xjgi) =Z x+0:5x¡0:5f(»)d»; (7)with zero location(mean) parameter. Using (6), the informa-tion value of the prediction error ei is defined asJi = ¡ log2 Prgi(eijB^(pi); gi): (8)Using (8), we optimize those thresholds to minimize in-formation values of the prediction error.3.3 Shape of DistributionError distributions of the prediction error of all quantiza-tion groups might become either a Gauss or Laplace distribu-tions. On the other hand, the measured in a classified groupsmight have a different distribution.In case of the design of predictor using MMSE, we de-sign it so as to make large prediction errors small. As a result,very small prediction errors are disturbed. Thus, the distri-bution will be close to a Gauss distribution and the codingperformance will be improved assuming the error distribu-tion is Gaussian. So, we expect that when we estimate theinformation value of the MMSE error, Gaussian distributionwill be good selection for model PDF.On the other hand, in case of the design of predictor usingMMAE, we resort to a linear programming using Barrodale’sTable 2: The Laplacian and the Gaussian function.Laplacian GaussianPDF 12be¡ jx¡¹jb 1p2¼¾e¡(x¡¹)22¾2locationparameter¹ = mediani(xi) ¹ =Pni=1 xinscaleparameter b =Pni=1 jxi¡¹jn ¾ =qPni=1(xi¡¹)2nmethod. As a result, at least k prediction errors become 0 andthe predictors are designed so as to become as many as pos-sible near 0[5]. Thus, the error distribution will be close to aLaplace distribution and the coding performance will be im-proved assuming the errors have a Laplace distribution forthe design of predictor used MMAE. So, we can expect thatwhen we estimate the information value of the MMAE er-ror, Laplacian distribution will be good selection for modelPDF. From the above discussion, it is advantageous to ap-proximate an actual distribution by Laplace distribution inMMAE coding.3.4 Distribution Shaping EffectFor design of predictor using MMAE, we know at leastk prediction errors become 0. In addition, k is inevitably re-lated to the accuracy of prediction, because this is the numberof pixels in the support region. If we design predictors usinga small k, the number of“0”prediction errors is decreased,and the accuacy of prediction is also decreased. Thus, theerror distribution will become as shown in Fig. 3-(a). On theother hand, if we design predictors using a large k, the num-ber of“0”prediction errors is increased, and the accuacy ofprediction is also increased. Thus, the error distribution willbecome as Fig. 3-(c).From the above discussion, we imagine that predictor or-der k may have a distribution shaping effect and that the cod-ing performance will be improved by shaping the error dis-tribution as Fig. 3-(b) using this effect.(a) k : small (b) k : pertinence (c) k : largeFig. 3: Distribution shaping effect of predictor order k. Thecoding performance may be improved by shaping the errordistribution using this effect.January 2009 Predictor Order and Error Distribution of MMAE Predictors for Lossless Image Coding954 SIMULATIONTo ensure that predictor order k has a distribution shapingeffect, we executed computer simulations.When we design predictors, the number of predictor classC and the number of iterations for block classification areimportant parameters. However, in order to prevent confu-sion, we fix C = 16 and the number of iterations is chosen10 in our computer simulation.We defined a fitting degree of model distribution(Laplacedistribution) to actual prediction error distribution in termsof the redundancy(the difference of real entropy and modelentropy) and measured the redundancies, changing the valueof k for several test images in Fig. 4.(a) ”airplane”(512£ 512) (b) ”baboon”(512£ 512)(c) ”camera”(256£ 256) (d) ”couple”(256£ 256)(e) ”lena”(512£ 512) (f) ”lennagrey”(512£ 512)(g) ”peppers”(512£ 512)Fig. 4: Test images used in simulation.The entropy of prediction errors I is calculated as fol-lows. When we denote the prediction error e’s event prob-ability pg(e) (e = ¡255;¡254; ¢ ¢ ¢ ; 255) in a group g, theentropy Ig of a group g is calculated asIg = ¡255Xe=¡255pg(e) log2 pg(e): (9)Then, when we denote the group g’s event probability Pg ,the average entropy I is calculated asI =NXg=1PgIg: (10)The model entropy of prediction errors I 0 is calculated as fol-lows. When we denote the prediction error e’s modeled eventprobability qg(e) (e = ¡255;¡254; ¢ ¢ ¢ ; 255) in a group g,the entropy I 0g of a group g is calculated asI 0g = ¡255Xe=¡255pg(e) log2 qg(e); (11)where pg(e) is the event probability of the prediction errore. Then, when we denote the group g’s event probability Pg ,the average model entropy I 0 is calculated asI 0 =NXg=1PgI0g: (12)Because I = I 0 when modeling is perfect, the real entropy Iis always smaller than the model entropy I 0;I <= I0:Thus, the smaller the the redundancy(= I 0 ¡ I), the closerthose distributions are to each other.Simulation results are shown in Fig. 5 in the next page.In Fig. 5, we can see that images may be divided into two cat-egories by the redundancy tendency. Redundancy of imagesin one category is increased in proportion to k like ”cam-era” and ”lena”, and that in the other category is decreasedlike ”airplane”, ”baboon”, ”couple”, ”lennagrey” and ”pep-pers”. Considering the vertical scale of figure, we can alsosee that this effect is more notable for small size images thanfor large size images. For some subimage of each category,error distributions in some quantization group when redun-dancy is maximum or minimum are shown in Fig. 6 in thenext page. Fig. 6-(a) and Fig. 6-(b) are the distributions of”camera” and ”couple”, respectively.In Fig. 6-(a), we can see that the model distribution isclose to an actual distribution when k is small. On the otherhand, in Fig. 6-(b), we can see that that is close to an actualdistribution when k is large. Thus, when we encode imageswhich have the similar characteristics as ”camera”, small or-der k prediction selection will be preferred. To the contrary,when we encode images which have the similar characteris-tics as ”couple”, the large k will be preferred.5 CONCLUSIONFrom Fig. 5 and 6, we see that predictor order k has adistribution shaping effect, and the coding performance willbe improved using the optimal k. However, the reason forrippling phenomena of Fig. 5 is under consideration.As the future work, we are due to investigate the codingsimulation using this effect. Also, when we design predictorsusing large k to use this effect, we should devise the methodto reduce side information of predictors.Hirokazu UENO and Yoshitaka MORIKAWA MEM.FAC.ENG.OKA.UNI. Vol. 43960.040.0420.0440.0460.0480.050.0520.0545 15 25 35 45 55predictor order kmodel redundancycameracouple0.020.0210.0220.0230.0240.0250.0260.0275 15 25 35 45 55predictor order kmodel redundancylenapeppers(a) ”camera” and ”couple”(256£ 256) (b) ”lena” and ”peppers”(512£ 512)0.020.0210.0220.0230.0240.0250.0260.0275 15 25 35 45 55predictor order kmodel redundancybaboonlennagrey0.0140.01420.01440.01460.01480.0150.01520.01540.01560.01580.0165 15 25 35 45 55predictor order kmodel redundancyairplane(c) ”baboon” and ”lennagrey”(512£ 512) (d) ”airplane”(512£ 512)Fig. 5: Simulation results. We see that images may be divided into two categories by the redundancytendency. Redundancy of images in one category is increased in proportion to k and that in the othercategory is decreased.00.010.020.030.040.050.060.07-30 -20 -10 0 10 20 30predictive error eevent probabilityactual distributionmodel distribution00.010.020.030.040.050.060.07-30 -20 -10 0 10 20 30predictive error eevent probabilityactual distributionmodel distribution(a-1) k = 51 (a-2) k = 7(a) ”camera”00.020.040.060.080.10.120.14-30 -20 -10 0 10 20 30predictive error eevent probabilityactual distributionmodel distribution00.020.040.060.080.10.120.14-30 -20 -10 0 10 20 30predictive error eevent probabilityactual distributionmodel distribution(b-1) k = 7 (b-2) k = 47(b) ”couple”Fig. 6: Error distributions(”camera” and ”couple”). We can see that there are the actual distribution is closeto the model distribution when k is large and it is close to the model distribution when k is small.January 2009 Predictor Order and Error Distribution of MMAE Predictors for Lossless Image Coding97REFERENCES[1] B. Meyer and P. Tischer, ”Glicbawls - Gray Level ImageCompression by Adaptive Weighted Least Squares,” Proc.Data Compression Conf. (DDC2001), p.503, Snowbird,Utah, 2001.[2] X. Wu, K.U. Barthel, and W. Zhang, ”Piecewise 2D au-toregression for predictive image coding,” Proc. of Int.Conf. on Image Process. vol. 3, pp. 901-904, 1998.[3] F. Golchin and K. K. Paliwal, ”Classified adaptive pre-diction and entropy coding for lossless coding of images,”Proc. of 1997 IEEE Int. Conf. on Image Process. vol. 3,pp.110-113, 1997.[4] Y. Hashidume, Y. Morikawa and S. Maki, ”MinimumMean Absolute Error Predictors for Lossless Image Cod-ing”, to appear in IEICE Trans. on Inf. & Sys. (D), vol.E91-D, no. 6, pp. 1783-1792, 2008.[5] I. Barrodale, F. D. K. Roberts, ”An Improved Algorithmfor Discrete l1 Linear Approximation”, SIAM Journal onNumerical Analysis, vol. 10, no. 5, pp. 839-848, 1973.Hirokazu UENO and Yoshitaka MORIKAWA MEM.FAC.ENG.OKA.UNI. Vol. 4398

Predictor Order and Error Distribution of MMAE Predictors for Lossless Image Coding

Abstract

Similar works

Full text

Available Versions

Okayama University Scientific Achievement Repository

Okayama University Digital Information Repository