Centre for Multimedia Signal Processing, Department of Electronic and Information EngineeringRefereed conference paper2000-2001 > Academic research: refereed > Refereed conference paperVersion of RecordPublishe

Chan, YH

Cheung, WF

PolyU Institutional Repository

OPTIMIZING MPEG-4 CODING PERFORMANCE BY TAKING POST-PROCESSING INTO ACCOUNT Wan-Fung Cheung and Yuk-Hee Chan Centre for Multimedia Signal Processing Department of Electronic and Information Engineering The Hong Kong Polytechnic University, Hong Kong ABSTRACT In this paper, a block-based coding scheme is proposed. Unlike those, schemes exploited in a conventional codec, the proposed scheme takes the post-processing scheme to be used in the decoder into account during encoding. An example is also provided in this paper to show how this coding scheme jointly optimizes the compression and the blocking-effect elimination when MPEG-4 deblocking filter is exploited in the post-processing stage. 1. INTRODUCTION Fig.2 Basic structure of the suggested imagelvideo coding system In low bit-rate image and video coding, block-based Discrete Cosine Transform (DCT) has been widely used. Noise caused by the coarse quantization of transform coefficients is noticeable as visible discontinuities among adjacent blocks in the reconstructed images. Various post- processing techniques such as block-boundary filtering [ 13, projections onto convex sets (POCS) [2,3], maximum a posteriori (MAP)' methods [4] and transform domain filtering [5,6] have been proposed to remove this blocking artifacts while maintaining the compatibility with current industrial standards. However, in these post-filtering schemes, attention has been focused on the elimination of blocking effect rather than on the optimization of the video codec. Figure 1 shows the general structure of current imagehideo encoding systems. In this structure, a post- processing is performed to a decompressed image in the receiver. However, this post-processing stage is not taken into account in the encoder so as to optimize the coding performance. We suggest modifying the basic structure of the encoding system as shown in Figure 2. By doing so, the compression and the post-processing processes can be jointly optimized in the encoder according tof-f'. In this paper, we will use an example to show how this idea works. We optimize a MPEG-4 [7] codec in a case that a deblocking filter is used in the decoder to eliminate the blocking effect such that the quality of the output will be better than that without optimization. 2. BLOCKING-EFFECT ELIMINATION In conventional schemes, the MPEG-4 encoder partitions the original image I into blocks of size 8x8, say Imn's, and then encodes the blocks with DCT block by block with a raster scan strategy. At the decoder side, the deblocking filter [8] performs one-dimensional filtering along the 8x8 block edges to eliminate the discontinuity among adjacent blocks. It is first applied to the horizontal edges and then the vertical edges. If pixel intensity is changed by a previous filtering operation, the updated pixel intensity is used for the next filtering operation. This deblocking filter has two separate filtering modes: DC offset mode and default mode. DC offset mode is applied in smooth region to smooth 8-pixel line segments which are perpendicular to the block's boundary, while default mode is applied in complex regions to smooth 2-pixel line segments. 3. ENCODER OPTIMIZATION We modify the MPEG-4 encoder according to the post- processing process to combine the compression and the restoration processes to make use of each other to optimize the overall coding performance. The philosophy of our approach is very simple. Instead of encoding block directly, we encode its modified version, say Y,,,, such that llF{ (T-'( Q{ T Here, Bmn denotes the greater area of a block named as Bmn, and, operators T{ .], Q{ .}, T-'{ .) and F{ .), respectively, perform the DCT transformation, the quantization, the inverse DCT transformation and the deblocking filtering [8] proposed for MPEG4 standard. The greater area of a block is actually the support region of { Y m , n }  1 I)*! - Im,n*II < IIF{(T'{Q{T(Im,n) 1 ))*I - Im,n*II. 965 0-7803-6297-7/00/$10.00 0 2000 IEEE Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:53 from IEEE Xplore.  Restrictions apply. operation F( .} performed to the block. The modification of I, is carried out with an optimization filter. Its design corresponds to that of the deblocking filter used in the post-processing stage to adjust the boundary pixels. Accordingly, it is one-dimensional and has two separate operation modes. In particular, it operates in Opt-smooth filtering mode to handle smooth region and Opt-defaultfiltering mode to handle complex region. Let V = (v,,v, ,v2 ,... v8, vg) be a particular pixel vector across the boundary of block C, and block Cml,, as shown in Figure 3, where C,,, is the MPEG-coded version of Y,,. At the decoder, this pixel vector will be post-processed with the one-dimensional deblocking filter suggested in [8]. The mean square deviation of the post-processing result, say 3 ,  from the corresponding pixel vector in the original image, say i , is then given by 5 =I[ (3  - f )  11' . The values of G2 which minimize 5 are determined by solving equations at/&, = O .  Note only pixels in the current processing block are used to minimize 5 .  Pixel vectors in the processing block are processed accordingly one by one in a predefined order until the whole desirable C,, is obtained. If a pixel was modified, its most updated value will be used as the input for the current processing. After obtaining the desirable C,, Y, can then be determined. In theory, Y, should be an array that makes C,, = T' IQ{ T{ Y,,} ] }. However, this cannot be practically achieved as quantization is involved and C,, may not be one of the possible DCT-coded outputs provided by the encoder. In our approach, we let Y, = Sometimes the proposed modification may not guarantee that F{ (TI( Q( T{ Y,,!} })*} is more faithful to I,,* than F( (T-'{ Q{ T{ I,,} } }) }. Hence, the encoder should determine whether Y, or I,, should be encoded. The selection is carried out by checking their post- processed results against the original image. The one provides the minimum mean square error (MSE) and less bit stream is then encoded and transmitted. We proceed to process and encode the next block until the whole image is encoded. - CBll,". Block boundary on a vertical edge Fig. 3 Block boundaries of 8x8 blocks 4. OPTIMIZATION ORDER In order to minimise the interference among the modification results, an Optimization filtering with Checker scheme (OPF-CK) is used to modify I,, to Y,,,,,. In this scheme, blocks are processed in an order as shown in Figure 4. We process and encode the blocks at the gray color area first and then white color area until the whole image is encoded. The optimization filtering starts at the left-lower corner of a block and moves clockwise around the block. Hence, we can modify all pixel vectors around the boundaries of the block. Besides, the blocks in the area of the same colour can be processed in parallel to reduce processing time as the modification of these blocks does not affect each other. As the modification made in a particular block will influence the deblocking result of some previously modified blocks next to it, the pre-processing is performed in an iterative way in order to reach an optimized output. From our simulation result, we found that it converged rapidly and was able to reach a nearly optimized rate- distortion value after 2 iterations. All results shown in the paper are obtained with 2 iterations unless it is specified. (Ar row shows the scanning path) Fig.4 Checker scans strategy 5. SIMULATION RESULT Computer simulation has been carried out to demonstrate that a corresponding pre-processing step can definitely improve the coding performance if we take the post- processing step into account. In our simulation, the MPEG- 4 video VM 9.1 coder [7] is used. We consider I-VOP and P-VOP with a fixed quantization parameter (QP). The ITU-T Recommendation H.263 [9] quantization method is adopted and the motion search range is [-16., 15.51. Various QCIF test sequences of size 150 frames were used to evaluate the performance of the proposed scheme. Only the first frame of a sequence is coded as intra-frame. Table 1 shows the simulation results of 5 testing video sequences. One can see that the PSNR and the compression rate can be further improved if a pre- processing step is performed prior to encoding and the proposed pre-filtering scheme is robust with respect to various video sequence characteristics. Figure 5 shows the PSNR improvement for Akiyo and Silenr sequences at a fixed QP. Though the improvement seems little in terms of absolute figure, the 966 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:53 from IEEE Xplore.  Restrictions apply. pre-processing step actually introduces a twofold increase in the improvement in general. Similar results were obtained with different testing video sequences and different sets of QP parameters. Figure 6 shows the rate- distortion performance of different approaches. One can see that the improvement is very respectable at low bit rates. Figure 7 shows the deblocking results for the Bream sequence. The proposed pre-processing scheme can preserve the details while reducing blocking artifacts. This can be observed by examining the texture detail around the anal fin and the tail of the fish in Figure 7. 6. CONCLUSION In this paper, a pre-processing method is proposed to improve the coding performance of MPEG-4 by jointly optimizing both compression and blocking-effect elimination. The simulation results show that the proposed method can improve image quality subjectively and objectively without changing the core structure and the bit- stream format of a MPEG4 codec. 7. ACKNOWLEDGEMNTS The work described in this paper was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region (Project No. PolyU 5092/98E) (b) Fig. 5 PSNR improvement of different coding schemes: (a) Akiyo sequence at QP=13 and (b) Silent sequence at QP=l I .  I Areraee Rate Distortion for Alii70 49 - 41 - 3 E 2 J9 a J4 - 29 - 0 01 1 Averaee Rate Distortion for Claire I 30 U 81 n i  I EltS Plrel (b) Fig. 6 Rate-distortion performance of different coding schemes: (a) Akiyo and (b) Claire. Average PSNR [bits] per frame I schemes. 8. REFERENCE [l] Y.H. Chan, S.W. Hong and W.C. Siu, "A practical post- processing technique for real-time block-based coding system," IEEE trans. on CAS for Video Technology, Vo1.8, No.1, Feb 1998, pp.4-8. [2] Y.Yang, N.P.Galatsanos, and A.K. Katsaggelos, "Regularized reconstruction to reduce blocking artifacts of block discrete cosine transform compressed images", IEEE Trans. CAS Video Technol., vol. 3 ,  no. 6, Dec 1993, pp. 421-432. [3] Y. Yang, N.P. Galatsanos, and A.K. Katsaggelos, "Projection-based spatially adaptive reconstruction of block-transform compressed images", IEEE Trans. CAS Video Technol., vol. 4, no. 7, Jul 1995, pp. 896-908. [4] T.P. ORourke and R.L. Stevenson, "Improved image decompression for reduced transform coding artifacts" lEEE Trans. CAS. Video Technol., vol. 5 ,  Dec 1995, pp. 490-499. 967 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:53 from IEEE Xplore.  Restrictions apply. S.S.O. Choy, Y.H. Chan and W.C. Siu, ”Reduction of block-transform image coding artifacts by using local [8] statistics of  transform coefficients”, IEEE Signal Processing Letters, Vo1.4, No. 1, Jan 1997, p.5-7. Z. Xiong, M.T. Orchard, and Y.Q. Zhang, “A deblocking algorithm for JPEG compressed images using overcomplete wavelet representations”, IEEE Trans. CAS. Video Technol., vol. 7, no. 2, Apr 1997, pp. 433-437. “MPEG-4 video verification model V.9.1,” ISODEC [9] JTCl/SC 29/WGIl/N2502a, Oct 1998. S.D. Kim, J. Yi, H.M. Kim and J.B. Ra, “A deblocking filter with two separate modes in block-based video coding”, IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 1, Feb 1998, ITU Telecom. Standardization Sector of ITU, “Video coding for low bit-rate communication,“ ITU-T Recommendation H.263. Mar 1996. pp. 156-160. . ,  Fig. 7 Coding results for Bream sequence (QP=12): (a) the 140th original frame; (b) encoded image of (a) (without pre-processing); (c) post-processing result of (b); (d) encoded image of (a) (with pre-processing); (e) post- processing result of (d). 968 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:53 from IEEE Xplore.  Restrictions apply. 

Optimizing MPEG-4 coding performance by taking post-processing into account

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository