Chan, YH

Hong, SW

Siu, WC

PolyU Institutional Repository

SUBBAND ADAPTIVE REGULARIZATION METHOD FOR REMOVING BLOCKING EFFECT Sung- Wai Hong Yuk-Hee Chan Wan-Chi Siu Department of Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong ABSTRACT This paper presents two new approaches to remove block- ing effect in low-bit rate transform coded images by using subband decomposition/reconstruction technique. They are designed to act as a supplementary post-processing step of the JPEG standard. Both approaches make use of the noise characteristic of each snbband to bound the maximum tol- erable error and the smoothness of the restored images in restoring snbband images with regularization. One of them will also utilize the spatial activity of the restoring images to tighten the bounds. Computer simulations showed that the new adaptive objective functions could achieve a bet- ter restoration performance in terms of both subjective and objective measures than did other conventional objective functions. 1. INTRODUCTION Discrete cosine transform (DCT) coding is a well es- tablished technique for image compression and has been adopted as the basic compression algorithm in the JPEG standard. In DCT compression, image is first divided into small non-overlapping blocks (usually 8 x 8 or 16 x 16) and each block is transformed with a 2-dimensional (2-D) DCT. Then, high-frequency (HF) coefficients of the transformed blocks are discarded, and the remaining low-frequency (LF) coefficients are quantized. However, block-based transform coding results in “blocking effect” [l, 21 at high-compression ratio. The blocking effect leads to the perception of visible discontinuity between adjacent blocks. In this paper, we present a spatially non-adaptive and a spatially adaptive subband approaches to remove this annoying artifact in the reconstructed images. These approaches are fully compati- ble with the JPEG standard. 2. REGULARIZATION APPROACH A linear space-invariant image degradation system can be modeled as ?/=Bf+n, (1) where vectors g ,f and n (lexicographically ordered by ei- ther column or row, from the size of N x N into N Z  x 1) are the degraded image, the original image and the additive This work was supported by The Hong Kong Polytechnic University Grant A/C No. 350.255.A3.420 0-8186-7310-9/95 $4.00 0 1995 IEEE 523 noise, respectively. Matrix B is the linear distortion opera- tor of size N 2  x N 2 ,  and y would be the blocky image that we reconstructed. In our case, B represents an operation which consists of the block DCT compression, quantization and decompression. Regularization [6, 71 is an effective technique to convert an ill-posed problem to a well-posed one. Ya.ng, Galatsanos, and Katsaggelos proposed an objective function (CLSI) [l] for tackling the blocking effect. In particulax, the objective function is defined as : where S is a regularizing operator of dimension N 2  x N 2 ,  which is generally a high-pass filter usedL to reduce the amount of noise (usually in the form of HF) in the restored image. 11 0 1 1  represents the Euclidean norm. Let c: and e: be the bounds for IIy - f 1 I 2  and llSf112, respectively, i.e., lly-f1I2 5 c:, and llSf112 5 6 ; .  The former bound is for the error that we can tolerate. With this constraint only, the solution obtained is an inverse filter which will amplify the noise during the restoration. Hence, satisfying the former constraint may have a side effect of noise amplification. The latter bound imposes a smoothness upperbound and sup- presses the HF content of the whole image by means of the regularizing operator. The principle of regularization is to find the best estimate that compromises these two con- straints. The ratio 011 (= 5)  is the regularization parame- ter that controls the degree of smoothness of the resulting image. A solution to the fore-mentioned problem can be ob- tained by minimizing the objective function (eqn.(2)) with respect to f. The iterative method, which has a number of advantages [8], is given by 4 where I is the identity matrix and X t  denotes the trans- pose of matrix X. The relaxation parameter /31 is a scalar, which has to be within the range 0 < p1 < to 11 + a1StSII ensure the convergence of the iteration. The iteration f(k) will converge to a unique estimate of the original image. Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:46 from IEEE Xplore.  Restrictions apply. 3. PROBLEM FORMULATION IN SUBBAND DOMAIN The basic technique of subband decomposition and re- construction [3] can be briefly explained as follows: The input image is decomposed into several narrow bands by passing through an analysis filter bank. Subband images are then sub-sampled for further processing. In reconstruc- tion, processed subbands are up-sampled and filtered for interpolation with a synthesis filter bank. Then they are recombined to form the reconstructed image. I t  is desirable to design the analysis/synthesis filter banks 131 in such a way that they can remove aliasing between subbands and achieve perfect reconstruction. For alias-free subband decomposition, the degradation model in subband domain [4, 51 becomes yz = B,f,  +n ,  f o r i  = 1, ... , M. (4) where y t ,  B,, fz, and nz are the sub-sampled observed im- age, the subband distortion operator, the original image, and the noise respectively of the zth subband. M is the number of decomposed subbands. Figure 1 depicts four subband images of a JPEG en- coded image (0.26 bit/pixel). The image is decomposed with an 8-tap perfect reconstruction-quadrature mirror fl- ters (PR-QMFs) [3]. Subband LL is very close to the JPEG encoded version. Subbands LH, HL and HH contain the horizontal, vertical and diagonal features of the original JPEG encoded image, respectively. The appearance of the horizontal and vertical line segments in subbands LH and HL are caused by the quantization process of the inde- pendent block-based transform coding scheme. It  is ob- vious that,  better images could be achieved by adapting the restoration models to the characteristic of each sub- band separately. In other words, we aim at adapting the regularization technique to the appearance of the blocking artifact of each subband. 4. SPATIALLY NON-ADAPTIVE SUBBAND APPROACH We propose the following new subband objective func- tions for a M-subband approach : where s E M subbands. In these functions, W i  and W i  are diagonal weighted matrices of dimension $ x g. Param- eter a:, S" ,  f" and y" are defined as before for a particular subband. There exists a trade off between computational load and the number of subbands. We have found that a 4- subband system is robust enough for this application, with an acceptable increase in computational load only. For a 4-subband system, we have s E A In highly compressed JPEG encoded natural images, most HF DCT coefficients are discarded. Therefore, in sub- band domain, the low-frequency subband (subband LL) will contain relatively large amount of signal energy than the mid-frequency (subbands LH and HL) and high-frequency subbands (subband HH). Usually, subband HH almost con- tains no signal energy. Hence, for subband HH, where the signal-to-noise ratio (SNR) is the lowest, noise suppression { LL, LH, HL, HH }. should have an overwhelming effect. Subbands LH and HL are with moderate SNRs, and characterized by the block- ing artifact, which is apparently observed as hori vertical lines segments. Therefore, noise suppressi be specifically strengthened in these segments, while mod- erate restoration condition is applied to other regions. most of the signal energy is retained in subband LL, similar restoration conditions should be applied as in the fullband image. Based on the above arguments, we adjust parameters W i  and W$ for particular subband. To simplify the weighted matrices determination, subband LL is taken as the reference for determining the weighted matrices for the other subbands. Hence, we have W i L  = W i L  = I .  For subband HH, we have WEH = 0 1  and W f H  = 71, where y 1 1, to remove the HF component amplification effect and enhance the noise suppression effect simultaneously. Sim- ilarly, for subbands LH and HL, where noise suppression strength can be approximated to be that in subband HH, we have Wi" = W f L  = W F H .  As for Wk" and they should be determined according to their noise cha teristics as follows : 0 p otherwise, if it is coincides with the line segments, "2 fo r i  = 1, ..., T .  where p is a scalar weight, with the range 0 < p < 1. Hence, we can rewrite eqn.(5) as : iwi"(a, i) OT W,""(Z, 2) = J: = a211SLLfLL112 + IlyLL - fLL112, J: = ya2IIS f II + IIw:H(YLH - fLN)1I2, Jo"," = Y'Y211S f  11 + IIWRHL(YHL - f H L ) l l 2 ,  JU"," = Y 4 I S  f II . (6) L H  L H  2 H L  H L  2 H H  H H  2 (9) Instead of looking for a solution whic four objective functions with respect to their CO subband images simultaneously, we minimi bination of the above equations with respect image. In formulation, we have Since it is very difficult to determine an optimal reg- ularization operator (S ' )  for each subband, an alternative objective function (SCLS1) given as 9L 82 Ja3 = mIIS [w;S"l IT + II [Wi3YC - f"1 I? (11) s € A  SEA 9L is exploited instead, where [e] is the reconstruction op- SEA erator which combines all four subbands to reconstruct an image with the synthesis filter bank. teration, instead of performing regularization to nd sepa- rately, the subbands predicted and the prediction error of each subband are respectively weighted by their specific W i  and W i  first. Then, the weighted terms are reconstructed into fullband domain before SCLSl is performed. Function SCLSl is spatially non-adaptive as it is based on the linear space-invariant image model (eqn.(l)). Since 524 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:46 from IEEE Xplore.  Restrictions apply. this model may simplify the nature of natural images, per- forming SCLSl on spatial varying images may not get a good result. In this case, modification of SCLSl is neces- sary to preserve the high spatial activity components (e.g. edges) of the restoring images. 5. SPATIALLY ADAPTIVE SUBBAND APPROACH In order to restore the sharpness of the restored images, a new spatial adaptive subband function (SCLS2) that em- beds the spatial activity information of the restoring images is presented. This function is defined as w w S E A  SEA (12) where R and L are both diagonal weighted matrices and defined as in [2], so as t o  compromise the effect of the con- straints with respect t o  the spatial activity [2]. Minimizing the objective function (12) with respect to f results in the following iterative equation : f(k 4-1) = f ( k )  +P4(RtRy - (Rt  R+a4St L t L S ) f ( k ) ) .  (13) 6. SIMULATION RESULTS In our computer simulations, three typical 256-level test images, ‘Baboon’, ‘Lena’ and ‘Hat’ were selected. They are a l l  of size 256 x 256. The spatial activity distribution of these images range from high to low [SI. The test images were divided into 8 x 8 blocks and transform-coded with JPEG scheme to  generate blocky images. The blocky im- ages were then restored by making use of CLSl (eqn.(Z)), as well asour proposed SCLSl (eqn.(ll)) and SCLS2 (eqn.(l2)), with initial estimates prepared by using the approach pro- posed in [2]. A 3 x 3 Laplacian filter was used as the regularization operator S. The regularization parameters were chosen to be (YI = (z)2 and a3 = a 4  N 4~x1. 8-tap PR-QMFs, which are designed to minimize the interband aliasing energy of the filtered signals [3], were used as the analysis/synthesis filter banks. The PPSNR was used as an objective criterion of merit. Criterion Il’k-’‘-1’la , l rk l ,a  5 2 x lo-” was used to terminate the iterative processes. The PPSNR is defined as where gr,3 and z ~ , ~  are the ( i ,  j ) t h  pixels of the original and the output images respectively. Table 1 lists the PPSNR improvements (IPPSNR) of the restored images and the number of iterations required to achieve these performance. It is obvious that our proposed functions yield better objective measures than CLSl does. On the average, the objective improvement of our proposed functions are approximately 3.4 times over that of CLSl in terms of dB. It  is easy to realize that,  SCLS2 yields the best objective measures among the three functions, for all test images. Figure 2 shows the magnified portion of a JPEG en- coded ‘Lena’ (PPSNR = 28.3573 dB). Figure 3, 4 and 5 are the magnified portions of ‘Lena’ after being processed by CLSI, SCLSl and SCLS2 respectively. With our proposed objective functions, the restored images are free from the blocking effect. Especially with SCLS2, the sharpness of the restored images is well restored and preserved. 7. CONCLUSIONS In this paper, we presented two new adaptive objective functions for the constrained least square regularization ap- proach to remove the blocking artifact. Findings reveal that the proposed objective functions performed better than the regularization approach proposed in [l], on both objective and subjective measures. 8. REFERENCES Y. Yang, N. P. Galatsanos and A. K .  Katsaggelos, “Regularized reconstruction to reduce blocking arti- facts of block discrete cosine transform compressed im- ages,” IEEE Trans. on Cercvats and Systems for Vzdeo Technology, Vol. 3, No. 6, pp. 421-432, Dec. 1993. S. W. Hong, Y. H. Chan and W. C. Siu, “An adaptive constrained least square approach for removing block- ing effect,” IEEE Internatzonal Symposzum on Czrcuzts and Systems, Vol. 2 ,  April. 1995. A. N. Akansu and R. A. Haddad, Multtresolution signal decomposztoon: transforms, subbands, and wavelets, Academic Press, Inc.. N. P. Galatsanos and R. T. Chin, “Digital restoration of multichannel image,” IEEE Trans. on Acoustacs, Speech, and Srgnal Processing, Vol. 37, No. 3, pp. 415- 421, Mar. 1989. M. R. Banham, N. P. Galatsanos, H. I,. Gonzalez and A. K. Katsaggelos, “Multichannel restoration of single channel images using a wavelet-based subband decom- position,” IEEE Trans. on Image Processzng, Vol. 3, K. Stewart and T. S. Durrani, “Constrained signal re- construction - a unified approach,” EURASIP Szgnal Processang 111, pp. 1423-1426, 1986. A. K. Katsaggelos and R. M. Mersereau, “A regu- larized iterative image restoration algorithm,” IEEE Trans. on Sggnal Processang, Vol. 39, No. 4, pp. 914- 929, Apr. 1991. A. I<. Katsaggelos “Iterative image iestoration algo- rithms,” Optical Engcneerang, Vol. 28, No. 7, pp. 735- 748, July 1989 V. R. Algazi, Y. Kato, M. Miyahara and Kotani, “Comparison of image coding techniques with a pic- ture quality scale,’’ Proc. SPIE Applrcataons of Degztal Image Processang xv, vol. 1771, pp. 396-405, 1992. NO. 6, pp. 821-833, NOV. 1994. 525 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:46 from IEEE Xplore.  Restrictions apply. a: subband LL size: $ x + JPEG encoded image ‘Baboon’ ‘Lena’ ‘Hat’ b: subband LH size: $ x + IPPSNR (# of iterations) SCLSl SCLS2 bit/pixel cLsl [11 0.33 0.084dB(13) 0.146dB(04) 0.151dB(08) 0.26 0.136dB(13) 0.490dB(07) 0.541dB(19) 0.28 0.210dB(14) 0.994dB(20) 1.037dB(20) c: subband HL size: $ x $ d: subband HH size: $ x + Figure 1: Four subbands of a JPEG encoded image. Figure 2:  JPEG encoded ‘Lena’ Figure 3: CLSl processed ‘Lena’. Figure 4: SCLSl processed ‘Lena’. Figure 5: SCLS2 processed ‘Lena’. 526 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 23:46 from IEEE Xplore.  Restrictions apply. 

Subband adaptive regularization method for removing blocking effect

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository