Centre for Multimedia Signal Processing, Department of Electronic and Information EngineeringRefereed conference paper2003-2004 > Academic research: refereed > Refereed conference paperVersion of RecordPublishe

Chan, YH

Fung, YH

Centre for Multimedia Signal Processing, Department of Electronic and Information EngineeringRefereed conference paper2003-2004 > Academic research: refereed > Refereed conference paperVersion of RecordPublishedVoR allowe

PolyU Institutional Repository

2004 IEEE International Conference on Multimedia and Expo (ICME) Restoring Halftoned Color-quantized Images with Simulated Annealing Yik-Hing Fung and Yuk-Hee Chan Centre for Multimedia Signal Processing Department of Electronic and Information Engineering The Hong Kong Polytechnic University, Hong Kong ABSTRA C7 Restoration of color-quantized images is rarely addressed in the literature especially when the images are color-quantized with halftoning. Direct applications of existing restoration techniques are generally inadequate to deal with this problem. In this paper, a restoration algorithm based on simulated annealing is proposed to solve the problem. This algorithm makes a good use of the available color palette and the mechanism of a halftoning process to derive useful a priori information for restoration. It is shown to be able to provide a good restoration result. 1. INTRODUCTION Color quantization is the process of reducing the number of colors in a digital image by replacing them with a representative color selected from a palette [1,2]. It is widely used nowadays as it lessens the burden of massive image data on storage and transmission bandwidth that are bottlenecks in many multimedia applications. When color quantization is performed, certain types of degradation are introduced due to the limited colors used to produce the output image. There are two most disturbing defects. One is the false contour which appears in smoothly changing regions and the other is the color shift in the color-quantized images. The smaller the color palette size, the more severe the defects are. Digital halftoning [3-61 would he helpful to eliminate these defects by making use of the fact that human eyes act as spatial low-pass filters. At the moment, the most popular halftoning method is error diffusion and several well-known error diffusion filters such as Floyd-Steinberg's filter [3] are generally used to achieve the goal. Color quantization is a kind of degradation to the original full-color image. Image restoration is sometimes necessary for one to recover the original image from its color-quantized version. However, though there are a lot of reported works on the restoration of noisy and blurred color images [7]-[ 141, little effort has been seen in the literature for restoring halftoned color-quantized images. Obviously, the degradation models of the two cases are completely different and hence direct adoption of conventional restoration algorithms does not work effectively. Simulated annealing [I51 is a general adaptive heuristic and belongs to the class of non-deterministic algorithm. As compared with some other conventional methods, it accepts solution with deteriorated cost to a limited extent. This feature gives the heuristic the capability to escape from the local minimum. This paper is devoted to formulating the process of color quantization when error diffusion is involved and 0-7803-8603-5/04/$20.00 2004 IEEE 367 developing a simulated annealing restoration algorithm to restore corresponding degraded images. 2. IMAGE DEGRADATION IN COLOR QUANTIZATION WITH HALFTONING A color image X generally consists of three color planes, say, X ,  , X g  and X b  , which represents the red, the green and the blue color planes of the image respectively. Accordingly, the color pixel of a 24-bit full color image of size N x  N consists of three color components. The intensity values of these three components form a 3D vector X ( ; j )  = (X(; j ) , ,X(;J)b . ,  X ( i j ) b ) ,  where X ( i j ) c  e [0,1] is the intensity value of the cth color component of the (ij)th pixel. Here, we assume that the maximum and the minimum intensity values of a pixel are, respectively, 1 and 0 .  Figure 1 shows the system which performs color quantization with crror diffusion. The input image X is scanned in a row-by-row fashion from top to bottom and left to right. The relationship between the original image X and the encoded image Y is described by - u(; , j )c  x ( i , j ) c  - ZH(~,Q(;-~, j - l ) c  %,j)  =Qc[u( i , j ) l  ( 2 )  (1) (k&S and t ( i . , )  = Y(; . j )  -U(;,,) = Q , [ f k . j ) I  - U U , ~ )  (3) where c(;,j) is a state vector of the system, E(; , j )  is the quantization error of the pixel at position ( i j )  and H(k,l)c is a coefficient of the error diffusion filter for the c' color component. S is the corresponding casual supported region of H(k , l ) c  ' The operator Q,[.] performs a 3D vector quantization. Specifically, the 3D vector f i ( ; , j )  is compared with a set of representative color vectors stored in a previously generated color palette C = { i-; : i=1,2, ... N ,  ). The hest-matched vector in the palette is selected based on the minimum Euclidean distance criterion. In other words, a state vector U p j )  is - Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:19 from IEEE Xplore.  Restrictions apply. represented by color G k  if and only if l l U ( i , , ) - ? k  112 IIU(i,j) - G j  / I  for all j=l,2, ... N, . Once the best-matched vector is selected from the color palettc, its index is recorded and the quantization error E( i , j ,  = C k  - U ( i , j )  is diffuscd to pixel (ij)'s neighborhood with eqn. ( I ) .  Note that, to handle the boundav pixels, E(i , j )  is defined to be zero when ( i ,  j )  falls outside the image. The recorded indices will be used in the future to reconstmct the color-quantized image with the same color palette. - - - Qch Figure 1. Color quantization with halftoning 3. FORMULATION OF THE PROPOSED ALGORITHM Let S he the output of the restoration. Obviously, the restored image S must be equal to Y if it is color-quantized with error diffusion. In formulation, we have where QJ-1 denotes the operator which performs color quantization with halftoning as shown in Figure I.  Accordingly, the cost function of a restored image can be defined as where Z[l] denotes the total number of nonzero elements in image I. In our approach, S is searched with the simulated annealing algorithm to minimize the cost function E. The simulated annealing algorithm is a double-loop iterative algorithm that simulates the annealing process at a given temperature T . The core part of the algorithm is the Metropolis procedure. During the iteration of simulated annealing algorithm, the temperature T is reduced in a controlled manner as given by where Tk is the temperature at outer iteration k and a is a constant used to achieve cooling. At a particular temperature Tk , the amount of time spent in annealing is gradually changed by Mk+l =pMk (7) where M k  is actually the number of inner iterations performed at temperature Tk and fl  is a constant used to do the adjusbnent. The algorithm is terminated at temperature T, when x;=, M k  is larger than a predefined threshold I, ,  . Here, we assume that the simulated annealing process starts at its initial y = Qch[Sl (4) E = z[y - '&h[sll ( 5 )  & + I  = ar, (6) stage k 0 .  Let S,,, be the current estimate of the restored image at a pamcular inner iteration at temperature Tk and E,,,, he its corresponding cost. The new estimate of the restored image is made with S,, by where y is a controlling parameter used to control the amount of perturbation applied to the S , ,  . The cost of S,,, , say, E,,, is then evaluated with eqn. (5 ) .  When E,,,<Ec,,, happens, S,,, is updated to be S,, . Furthermore, if E,,, < Ebel happens, where Eber, is the cost of the hest estimate so far ( S h e s , ) ,  then will be replaced by S, . In formulation, wc have Sn,=Scu,  +~(Y-Q~ds,.,l) (8) When E,,> E,,, happens, Metropolis will accept S,, by updating S, to be S,, if r < e(Emr-E'-)'Knr , where r is a randomly generated value which is uniformly distributed between 0 and I ,  T denotes the current temperature and KB is the Boltmann constant. This criterion for accepting the new solution is h o w n  as the Metropolis criterion. At the beginning, temperature T is high. This permits many uphill moves and provides chances for the solution to leave a local minimum. As temperature T is reduced gradually, fewer and fewer uphill moves are permitted and only downhill moves are allowed eventually. 4. SIMULATIONS Simulation has been carried out to evaluate the performance of the proposed algorithm on a set of color- quantized images. In our simulation, a number of de facto standard 24-bit full-color images of size 256x 256 each were used. The images were color-quantized to produce Y 'S. The color palette used for quantization was of size I28 colors and was generated with the median cut algorithm [2]. In color quantization, halftoning was performed with error diffusion and the Floyd-Steinberg diffusion filter [3] was used. The proposed restoration algorithm was applied to restore the halftoned color- quantized images (Y 's) In the realization of the proposed algorithm, both Su , the initial estimate of S , and She, were initialized to be the filtered output of the observed image Y . Specifically, a 3 x 3 Gaussian filter was used to generate So and Shest . Parameter Tu was selected to be I E,, -Es., 1 / ( K B  log(0.95)) , where Eso and Eso were, respectively, the cost of So and S', . Here, S', is the first estimate obtained with eqn. (8) based on Su . This allows reasonable amount of uphill move at the beginning. Parameter a was selected to be 0.9, the middle value of the 368 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:19 from IEEE Xplore.  Restrictions apply. selection range suggested in [IS]. To simplify the algorithm, we selected p to be I such that we had M k  = M for all k 2 0 .  Different combinations of M and y were evaluated to study their impact to the restoration performance. Figure 2 shows the simulation results of restoring the color-quantized “Lenna” with different combinations of M (=IO, 50 and 100) and y (=0.001, 0.002, 0.005, 0.015, 0.05, 0.2). Similar results were obtained with some other testing images. Here, SNRI is defined as where .??ci,j), ?li,j) and sbulli,j) are, respectively, the ( i , j ) I h  pixels of the original, the color-quantized and the restored images. In Figure 2, the dotted lines correspond to the cases of M I 0  while the solid lines correspond to the cases of M 5 0  and 100. For a pmicular y , the curve of M50 overlaps with the curve of MIOO. When y > 0.002, using different values of M makes very little difference. Another observation we have is that, when M250 , the smaller the value of y , the more iterations are required for the estimate to converge but the better SNRI performance the converged output can be achieved. Based on the evaluation results we have, it can be found that, for a selection of I , ,  =lOOO, the combination of M =50 and y =0.002 is a reasonable choice for restoring color- quantized images. With this combination, the best SNRI performance can he achieved in around 1000 iterations. It is sufficient for the estimate of the iterative algorithm to converge. If complexity is a critical concern, one may select a combination of M =SO and y =0.05, which can achieve a SNRl close to the one obtained with the previous selection in less than 100 iterations. In this paper, the presented results of the proposed algorithm were obtained with a =0.9, p = I ,  To = 1 Eso - I / ( K B  log(0.95)) , f , ,  = I  000, M = 50 and For comparison, some other restoration algorithms that were originally proposed for restoring noisy and blurred color images were also been evaluated. They were simulated here for comparative study as few schemes had been proposed for restoring color-quantized images and they are typical examples of the type ([7]-[14]). In particular, Galatsanos’s algorithm [9] is based on the constrained least square approach and Hunt’s algorithm [IO]  is based on Wiener filtering. There were two algorithms presented in Alhmbasak’s work [ I  I]. The first one makes use of the correlation among the color components of a pixel while the other one does not. They are, respectively, referred to as KL and IND in [ I  I]. Both algorithms take the colorimetric aspects into account and hy to minimize the error in CIELAB space [16]. In realizing Galatsanos’s algorithm [9], the noise power of each channel was estimated with the original full-color image. In realizing Hunt’s algorithm [IO], three separate Wiener filters were used in three different channels and, during the design of the filters, the noise spectrum of each channel was estimated with the original full-color image. Similarly, the original full- y =0.002. color image was used to estimate the power spectra of different channels in realizing Altunbasak‘s algorithms [ I  I]. In a practical situation, no original image is available and hcnce all mnst be estimated from the degraded image. In other words, in practice, the restoration results of [9], [IO] and [ I l l  may not as good as those presented in this paper. As it is not necessary to use the original full-color image to extract information for the proposed algorithm, additional credit should be added to the simulation results of the proposed algorithm indeed. Table I shows the SNR improvement (SNRI) achieved by different algorithms. From Table 1, one can see that the performance of the proposed algorithm is better than that of the others. On average, with the proposed algorithm, a SNRI of 7.89 dE3 in image quality was achieved for images color-quantized with a palette of size 128. Tables 2 and 3 show the performance of the evaluated algorithms in terms of the CIELAB color difference (AE) metric. A well accepted rule of thumb is that color error is visually detectable when A E > 3  [11][17]. Table 2 shows the average of the AE values of all pixels in a restoration output and Table 3 shows the percentage of pixels whose color error is visually undetectable in a restoration output. Again, one can see that the proposed algorithm is superior to the others. 5. CONCLUSIONS In this paper we have introduced a restoration algorithm for restoring halftoned color-quantized images. This algorithm makes a good use of the available color palette and the halftoning process to derive useful a priori information for restoration. Significant improvement in various aspects can be achieved as compared with other conventional algorithms. 6. ACKNOWLEDGEMENTS This work was substantially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region (Project No. PolyU 5217/038) and a grant from the Centre for Multimedia Signal Processing, HKPolyU (A046). REFERENCES [I] M. T. Orchard and C. A. Bouman, “Color quantization of images,’’ IEEE Tram. on signal Processing, Vo1.39, No.12, Dec 1991, pp.2677-2690. P. Heckbert, “Color image quantization for fnme buffer displays,” Compur. Graph., vol. 16, no. 4, 1982, pp. 297- [2] 3n7. .. [3] R. Ulichney, Digital Howoning, Cambridge, M A  MIT PIP4C 19x7 . .-~., .- .  . [4] R. S. Gentile, E. Walowit, and J. P. Allebach, “Quantization and multi-level halftoning of color images for near original image quality,” Proc. SPIE, vol. 1249, 1990, pp. 249259. [SI S. S. Dixit, “Quantization of color images for displayiprinting on limited color output devices,” Compuf. Graph., vol. 15,110.4, 1991, pp. 561-567. X. Wu, “Color quantization by dynamic programming and principal analysis,’’ ACM Trans. Graph., vol. I I ,  N0.4, 1992, pp. 384-372. [6] 369 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:19 from IEEE Xplore.  Restrictions apply. [7] M. Bami, V. Cappellini and L. Mim, “Multichannel m- filtering for color image restoration,” Proc., IEEE ICIP’ZOOO, Vol. 1,2000, pp. 529-532. G. Angelopoulosnd I. Pitas, “Multichannel wiener filters in color image restoration,” IEEE Trans. on CASYT, Vo1.4, No.1, Feb 1994, pp.83-87. N. P. Galatsanos, A. K. Katsaggelos, R. T. Chin and A. D. Hillery, “Least squares restoration of multichannel images,” IEEE Tram. on Signal Processing, Vo1.39, No.10, Oct 1991, pp.2222-2236. [IO] B. R. Hum, and 0. Kubler, “Karhunen-Loeve multispectral image restoration, Part 1: Theory,” IEEE Trans. on A S P ,  Vo1.32,No.3, Jun 1984.pp.592-600. [ l l ]  H. Altunbasak, and H. J. Trussell, “Colorimetric restoration of digital images,” IEEE Trans. on Image Processing, Vol.lO, No.3, Mar 2001, pp.393402. [I21 K. J. Boo, and N. K. Bose, “Multispectral image restoration with multisensor,” IEEE Trans. on Geoscience and Remote Sensing, Vo1.35, No.5, Sep 1997, pp.1160- 1170. [I31 N. P. Galatsanos and R. T. Chin, “Digital restoration of multichannel images,” IEEE Trans. on ASP, Vo1.37, Mar [14] N. P. Galatsanos and R. T. Chin, “Restoration of color images by multichannel Kalman filtering,” IEEE Trans. on Signa/ Processing, Vo1.39, No.10, Oct 1991, pp.2237- 2252. [I51 S. Kirkpatrick, C. D. Gelatt and M. P. Vecchi, “Optimation by Simulated Annealing”, Science, vol. 220, 1983, pp. 671-680. [I61 C.I.E. (1978) Recommendations on uniform color spaces, color difference equations, psychometric color terms. Supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971/(TC-1.3.). [17] C. Connolly, T. W. W. Leung and J. Nobbs, “Colour measurement by video camera: JSDC, vol.lI1, 1995, pp. 373-375. [8] [Y ]  1989, pp. 41 5-47. I .  Table I ,  SNR Improvements of various algorithms 5 - 5 a 0 a Figure 2. SNR Improvements achieved with various com- binations of y and M in restoring halftoned color- quantized “Lenna” Table 2. CIELAB difference AE measurement of the outputs of various algorithms Table 3. Percentage of pixels whose CIELAB difference AE is less than 3 after restoration 370 Authorized licensed use limited to: Hong Kong Polytechnic University. Downloaded on July 27, 2009 at 22:19 from IEEE Xplore.  Restrictions apply. 

Restoring halftoned color-quantized images with simulated annealing

Abstract

Similar works

Full text

Available Versions

PolyU Institutional Repository