Chan, YH

Hong, SW

Siu, WC

PolyU Institutional Repository

A PRACTICAL REAL-TIME POST-PROCESSING TECHNIQUE FOR BLOCK EFFECT ELIMINATION Sung- Wai Hong 1”uL-Hee Chan Wan-Chi Szu Dept. of Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong FAX : (852)2362-8439 TEL : (852)2766-6224 enhawaz@hkpuOl.polyu.edu.hk enyhcha~~@hkpucc.polyu.edu.hk enwcszu@hlcpucc.polyu.edu.hk ABSTRACT or even impossible to integrate them with the current industrial standard. The other strategy is to restore or enhance the im- age quality by Using post-processing techaiqL1es. T W  ical examples include block-boundary filtering [5] and convex projection (POCS) [6-lo]. This strategy has ap- peal since it only requires the decompressed image and is fully compatible with the existing JPEG standard. However, though most of them were proved to be very effective to eliminate the blocking effect, there are still some limitations. For instance, projection methods are not suitable for real-time application since they are In this paper, a non-iterative post-processing method is proposed to restore the images encoded with block-based compression standards such as JPEG. Tllis method classifies small local boundary regions accord- ing to their intensity distribution and then select appro- priate linear predictors t o  estimate the corresponding boundary pixels in the regions. This approach is easy to be implemented and we found in our simulations that its restoration performance was very respectable compared with the reported post-processing methods. 1. INTRODUCTION basically iterative algorithms which typically requires exhaustive computation and takes time for the estimate to converge to a desirable solution [6-lo]. On the other hand, some methods aim at the enhancement of the image quality instead of the restoration of the image and therefore the distortion may even be enlarged after the process [5, 71. In this paper, a non-iterative post-processing method is proposed to restore the JPEG-encoded images. This method classifies sinall local bound- ary regions according to their intensity distribution and then select appropriate linear predictors to  estimate the corresponding boundary pixels in the regions. This approach is easy to be implemented and we found in our simulations that its restoration performance is very re- spectable compared with the reported post-processing methods [5-lo]. The Joint Photographic Experts Group (JPEG) image compression standard [I] has been widely used as an industrial standard for still image compression. In this standard, source images are first divided into non-overlapped blocks of size 8 x 8 each. Each block is then transformed with discrete cosine transform (DCT). Transform coefficients are quantized and en- coded with arithmetic or Huffmaii coding technique. Since it is basically a block-based process, none of the inter-block correlation is exploited during the com- pression and therefore it causes visible discontinuities among adjacent blocks. This blocking effect is gener- ally considered to  be the most noticeable artifact in the reconstructed image. Two common strategies have been used to reduce the blocking effect. One tries to retain some informa- tion of the inter-block correlation in the transmitted message. This can he done by embedding inter-block correlation in the transform coeecients [2-31 or by directly sending extra information to the receiver via additional channel bandwidth [4, 51. Some of these methods can achieve a remarkable performance in elim- inating the blocking effect. However, since these meth- ods generally have to modify the basic transform coding structure of the existing JPEG standard, it is difficult This work was supported by The Hong Kong Polytechnic University Grant A/C No. 350.255.A3.420 0-7803-3258-X/96/$5.00 0 1996 IEEE 21 2. ALGORITHM To a certain extent, the elimination of the blocking ef- fect can be viewed as a process of image restoration. This restoration process involves the modification of the JPEG-encoded image. When there is no extra information about the original image, any modification is solely based on assumptions. In other words, post- processing could cause degmdation instead o i  improve- ment in the image quality. To minimize the possibility of the occurrence of degradation, it is reasonable to modify as few image pixels as possible during the post- Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:44 from IEEE Xplore.  Restrictions apply. processing process. Since the blocking effect appears as the discontinuity among adjacent blocks, it would be a good idea to smooth the discontinuity by modifying the intensity of the boundary pixels. However, as for the intra-block pixels, it is better to leave them without modification in order to  minimize the disturbance to the image pixels. It is well-known that the blocking effect is a consequence of ignoring the inter-block correlation dur- ing the compression. Therefore, one of the best ways to restore the original image is to make use of the residue inter-block correlation in the JPEG-encoded image to  recover the block boundary pixels. In the pro- posed method, we investigate the connection between the residue inter-block correlation in the encoded image and the boundary pixels in the region concerned, and then make use of the statistics obtained to  predict the original pixel intensity. Since there is typically a considerable amount of residue inter-block correlation remaining in the JPEG-encoded images, a good predic- tion result can be obtained based on the statistics even though only a simple linear prediction is performed. Consider an $-pixel line segment which is perpen- dicular to the block boundary as shown in figure 1. This segment is composed of two block boundary pixels and six neighboring pixels in line with them, which forms a pixel vector. Without losing the gen- erality, we assume that the pixel vector obtained after JPEG-encoding is v’= ( V O , V ~ ,  w2, w 3 , ~ 4 , 0 5 ,  ‘ugl ~ 7 ) .  Our objective is to predict the original boundary pixels p3 and p4 as accurately as possible. This can be done with linear prediction by making use of the intensity cor- relation among the original boundary pixels and their neighboring pixels in the JPEG-encoded image. Assume that the predicted value p3 is given by j = O  where aJ ’s  are scalar prediction filter coefficients, v(1 = v3 - m and m = iC:=,v,. The selection of the filter coefficients should be based on the least-squares minimization criteria In other words, one has to  min- imize the prediction error J = E[e;] = E[(p3 -p3)2] in an average least-squares sense, where E[.] is the expectation operator. The minimization of J can be achieved by satisfying the criteria dJ = 0 for all U , ,  which results in the following formulation aa, = EL,,,, Prediction coeffici its aj ’s can then be determined pro- vided that R;;), exists. The prediction filter for p4 can also be developed in a similar way. However, to avoid doubling the number of filters required, pixel 274 can be predicted by process- ing the reversed vect,or v‘: = (v7, 216, ~ 5 ,  w 4 , ~ 3 , ~ 2 , ~ 1 ,  V O )  instead of v’ with the same prediction filter. To assure the robustness of the prediction, pixel vectors a.re classified into different classes such that dedicated prediction filter can be designed based on the statistics of particular class of pixel vectors by making use of the above technique. As most -of the inter-block correlation can be reflected by the smoothness of the intensity distribu- tion across the block boundary, pixel vectors should be classified according to their intensity variation. One of the simplest but most effective way to achieve this objective is to make use of the concept of mean removal vector quantization. Consider the case that there is a codebook contain- ing N codewords C = {G : i = 1 , 2 , .  . ., N}, where N is the number of classes we are going to classify the pixel vectors. For any given mean-removed input pixel vector, classification is carried out by performing a VQ encoding procedure. That means one has to select a codeword which is closest to the given mean-removed pixel vector. The selection is based on the minimum Euclidean distance criterion, where the Euclidean dis- tance measure is defined as d = Cj,o(ca,j - and ci,j is the j t h  element of codeword 6 .  The index i E r block boundaly pixel \ blockbounda block corner pixel pixel vector /Hmllontal) Figure 1: Pixel vectors generated from images under JPEG coding. 22 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:44 from IEEE Xplore.  Restrictions apply. {1 ,2 , .  . . , N }  of the chosen codeword is then equiva- lent to  the class number with which the pixel vector associates. The codewords have to be carefully chosen such that they are representatives of their corresponding classes of pixel vectors. As usual, the codebook has to be designed with a set of training vectors prior to its appli- cation. Typical JPEG-encoded images can be used to generate training vectors. In particular, all vertical and horizontal mean-removed pixel vectors of the images, except those containing block corner pixels, are used as training vectors. The reversed vector of each train- ing vector is also used as a training vector to  double the size of the training set. Then, the LBG algorithm [ll] is used to  generate the optimal codebook with the set of training vectors. To assure the effectiveness of the post-processing method, the training set should be large enough and able to represent the statistics of the images to  be coded. The design of the codebook and the prediction filters could be computation-intensive, but their real- izat,ion can be off-line and no second realiza- tion is required once it is done unless the nature of input images is greatly changed. In real-time applica- tion, the proposed method consist of two stages. Fig- ure 2 shows the block diagram of the proposed post- processing method. After a JPEG-encoded image is decoded, pixel vectors of the decoded image are mean- removed and then classified with a VQ encoder. Based on the output of the VQ encoder, an appropriate pre- diction filter is selected for each mean-removed pixel vector. This vector is then fed into the filter to get a prediction value. The boundary pixel predicted will then be assigned the rounded result of the prediction value if it is not a block corner pixel. Note the reversed vector of the processing pixel vector is also a pixel vector and it gives another boundary pixel when it goes through the same procedures mentioned above. As for those block corner pixels, they are restored at  the end of the process. Since there will be two prediction val- ues (One from a horizontal pixel vector and one from a vertical pixel vector), the reconstructed pixel value is assigned to be the rounded result of the average value. Note that the pixel vectors used for restoring block corner pixels are composed of the restored boundary pixels. 3. SIMULATIONS Computer simulations have been carried out to evalu- ate the restoration performance of the proposed method. The simulations are based on a set of 256 level grayscale digital images of size 256 x 256 pixels. The blocky ver- class VQ encoder vedormean. m Figure 2: Block diagram of t,he post-processing method. sions of five standard images “Lenna” , “House”, “Pep- pers”, “Airplane” and “Sailboat” were used to generat,e a training set to obtain codebooks of size N ,  where A7 = 8, 16 and 32. These blocky versions were generated by coding the original images with the JPEG scheme at a compression rate around 0.27 bit per pixel. Training pixel vectors were then classified into N classes with the corresponding codebook. Prediction filters of var- ious classes were then designed with training vectors of their corresponding classes by using the approach mentioned in the previous section. The performance is evaluated in terms of the im- provement in signal-to-noise ratio which is defined as where FR, FB and FI are the restored , the JPEG- encoded and the original images respectively. Some other post-processing methods [5-71 for blocking effect elimination are also evaluated for comparison. The termination critelion for iterative methods is chosen to be l lFR,k-FR,k-- l l l  5 10WS, where F R , ~  is the restored image obtained at the kth iterat,ion. Table 1 shows the restoration performance of var- ious methods. One can see the superior restoration performance of the proposed method compared with other post-processing methods [5-71. In general, meth- ods using low-pass filtering [5, 71 may reduce the SNR since what they concern is to enhance the subjective quality by smoothing the block boundary. The modi- fied JPEG decoder proposed in ref [la] is efficient and practical, but its performance was much more sensitive than that of the proposed method to the choice of the training set. This can be seen from the fact t8hat, with the same small number of training images, its restora- tion performance fluctuated from training images to non-training images while the proposed method could provide a much more stable performance in our simulation. Table 1 also shows the performance of the proposed method with various number o€ classes. More classes llFR,k112 23 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:44 from IEEE Xplore.  Restrictions apply. Table 1: Restoration performance of various JPEG post-processing methods in terms of SNR improvement. t Images that were used for training in the simulation of method [12] and the proposed method. * It is modified such that the filtering output is bounded. This is also the first iteration result of [7]. resulted in better performance. However, 8 to  16 classes would be appropriate since it made the design sim- pler while maintaining a reasonable restoration perfor- mance compared with other reported methods [5-101. 4. CONCLUSION Based on linear prediction and vector quantization con- cepts, an effective post-processing technique for trans- form coding is proposed in this paper. This technique is fully compatible with the existing JPEG standard and enables real-time application. Computer simu- lations shown that the proposed method can achieve a better restoration performance in terms of SNR improvement compared with some other existing post- processing methods [5-81. Moreover, the image quality can also be improved subjectively by eliminating most of the blocking effect. 5. REFERENCES W. B. Pennebaker and J .  L. Mitchell, JPEG stall image data compressaon standard, Van Nostrand Reinhold, New York, 1993. H. S. Malvar and D. H. Staelin, “The LOT: Trans- form coding without blocking effects.” IEEE Trans. Acoust., Speech, Signal Processing, 37(4), pp. 553-559, Apr. 1989. Y. H. Chan and W. C. Siu, “An approach to  sub- band DCT image coding,” J .  Visual Comm. and Image Rep., 5(1), pp. 95-106, Mar. 1994. Y. Q .  Zhang, R. L. Pickholtz and M. H. Loew, “A new approach to  reduce the blocking effect of transform coding,” IEEE Trans. Comm., 41(2), pp. 299-302, Feb. 1993. H. C. Reeve and J .  S. Lim, “Reduction of blocking effect in image coding,” Opt. Eng., 23(1) ,  pp. 34- 37, Jan./Feb. 1984. Y. Yang, N. P. Galatsanos and A. IC. Katsagge- los, “Regularized reconstruction to reduce block- ing artifacts of block discrete cosine tra.nsform compressed ima,ges,” IEEE Trans. Cir. Sys. for Video Tech., 3(6), pp. 421-432, Dec. 1993. A. Zakhor, “Iterative procedures for reduction of blocking effects in transform ima,ge cocking,” IEEE Trans. Cir. Sys. for Video Tech., 2(1), pp. 91-95, Mar. 1992. T. P. O’Rourke and R. L.  Stevenson, “Improved image decompression for reduced transform coding artifacts,” SPIE, 218‘2, Image SL Video Processing S .  Minami and A. Zakhor, “An optimization a.p- proach for removing blocking effects in transforin coding,” IEEE Trans. Cir. Sys. for Video Tech., S. W. Hong, Y. H. Chan and W. C. Siu, “An adaptive constrained least square approach for re- moving blocking effect,” Proc. IEEE ISCAS’95, 2, pp. 773-776, Apr./May 1995. Y. Linde, A. Buzo and R. M. Gray, “A.n algorithm for vector quantizer design,” IEEE Trans. Comm.? S. W. Wu and A. Gersho, “Improved decoder for transform coding with applic,atioii to the JPEG baseline system,” IEEE Trans. Cornm., 40(2), pp. 251-254, Feb. 1992. 11, pp. 90-101. 1994. 5(2), pp. 74-81, Apr. 1995. 23, pp. 84-95, 1980. 24 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:44 from IEEE Xplore.  Restrictions apply. 

A practical real-time post-processing technique for block effect elimination

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository