Huang, Xu

Madoc, Allan

English

A maximum likelihood for Bayesian estimator based on /spl alpha/-stable is discussed. Closer to a realistic situation, and unlike previous methods used for the Bayesian estimator, for the case discussed here it is not necessary to know the variance of the noise. The parameters relative to Bayesian estimators of the model built up are carefully investigated after a discussion of /spl alpha/-stable 3D simulations for a maximum likelihood. The Bayesian estimator then is established. As an example, an improved Bayesian estimator that is a natural extension of the Wiener solution and other wavelet denoising (soft and hard threshold methods), is presented to illustrate our discussion.<br/

University of Canberra Research Repository

MAXIMUM LIKELIHOOD FOR BAYESIAN ESTIMATOR BASED ON &STABLE FOR IMAGE XHuank andA.C.Mado2 School of Electronics and Telecommunications Engineer ing ,  University of Canberra ,  ACT 2601, Australia Center for Actuarial S tudies ,  Department of E c o n o m i c s ,  University of Melbourne, VIC 3051, Australia 1 2 ABSTRACT A maximum likelihood for Bayesian estimator based on &stable is discussed. Closer to a realistic situation, and unlike previous methods used for Bayesian estimator, for the case discussed here it is not necessary to know the variance of the noise. The parameters relative to Bayesian estimators of the model built up are carefully investigated after a discussion of &stable 3-D simulations for a maximum likelihood. The Bayesian estimator then is established. As an example, an improved Bayesian estimator that is a natural extension of the Wiener solution and other wavelet denoising (sol? and hard threshold methods), is presented to illustrate our discussion. 1. INTRODUCTION Wavelet transform as a powerful tool for recovering signals from noise has been of considerably interest 11-71. In fact, wavelet theory combines many existing concepts into a global framework and hence becomes a powerful tool for several domains of application. Those methods that compute the correlation between coefficients at successive scales are based on the assumption that regular signal features show correlated coefficients at different scales, whereas irregularities due to noise do not 121. As mentioned by Achim el al. 181, there are two major drawbacks for thresholding. One is that choice of the threshold is always done in an ad hoc manner: another is that the specific distributions of the signal and noise may not be well matched at different scales. Donoho gives some minimum thresholds for several threshold schemes, titled "universal thresholds". These explicitly depend on the standard deviation of noise. where the standard deviation is assumed to be known. In practice, the standard deviation can be readily estimated using the methods discussed in 131, [9]. For some applications the optimal threshold can be computed. An approach different from "universal thresholds" is presented by Weyich and Nason I l O l .  in which cross-validation is used. Two approaches to cross validation are used. namely ordinary cross validation (OCV) and generalised cross validation (CCV): each is used to minimize the least-squares error between the original (which is the unknown value) function and its estimate based on the noisy observation. Several groups working on multispectral image restoration 1111, multichannel image restoration I121, and multiframe image restoration [13], have explored the Wiener filter. It i s  well known that a classical solution that deals with the noise removal problem is  the Wiener filter [14]. [15]. 0-7803-7304-9/02/$17.00 C2002 IEEE 709 However this method. designed mainly for additive noise suppression, has its limitations. Modelling the statistics of natural images is a challenging task because of the high dimensionality of the signal and the complexity of statistical structures that are prevalent. Numerous papers discuss modelling the statistics of nature images, including Bayesian processing presuppose proper modelling for the prior probahilitydensity function ofthe signal [8, 14, 15.16, 17. 181. In this paper the maximum likelihood for Bayesian estimator based on a-stable for image is discussed, and a method is offered such that it removes noise images without information about the noise- variance, even though this is one of key-parameters for normal Bayesian estimators. 2. ALPHA-STABLE DISTRIBUTIONS AND MAXIMUM LIKELIHOOD Tsakalides el al. [191. Achim el d. 18) and Peterson et al. I201 recently showed that alpha-stable distributions. a family of heavy tailed densities. are sufficiently flexible and rich to appropriately model wavelet coefficients of images in denoising applications. In this section we shall investigate maximum likelihood for an alpha-stable distribution after briefly introduce symmetric alpha- stable distribution (Sas). It is well known that the Sa5 distribution is defined by its characteristic function: @(U) = exp(j6w - Y I w 1% (1) The parameters a and 6 describe completely a SaS distribution. The characteristic exponent acontrols the heaviness of the tails of the stable density. a c a n  take values in (021: while a = 1 and 2 are the Cauchy and Gaussian cases. There is not closed-form expression known for the general SaS probability density function (PDF). The dispersion parameter y (y>O) refers to the spread of the PDF. The location parameter 6 i s  analogous to the mean of the PDF. Figure 1 shows the PDF for alpha-stable with different alpha, 1.0 to 2.0 when gamma takes 0.3. For comparison with the Laplace distribution, which was described in [I81 with s =1.5 and p = 0.8 is included in Figure 1. It is clearly shown that the smaller avalue implies heavier tails and Laplace distribution is significantly different from that of alpha-stable distributions. It is well known that if it is possible to minimize the mean square error (MSE). which is one of our major targets for denoising, when the bias is zero, and then the variance is also minimized. Such estimators are called "minimum variance unbiased" estimators, and they attain an important minimum bound on the variance of the estimator. called minimum variance bound. Figure 1: PDF for alpha-stable with (from top to down a = 1.0. 1.5 1.8. and 2.0 and Laplace distribution. If a variable 6' is unbiased it follows that - E ( 6 - 8 )  = O  (2) which can be expressed as: (3) -- where W )  = [xI (5 ) .x2 (5 )  ,...,x N ( n l T  and fkB  (% 8) is the joint  density of X'(8 , which depends on a fured but unknown parameter Following [18] we have The function In fR.(%@) is well known as the "log likelihood" function of B (LLF). Its maximum likelihood estimate can be obtained from the equation: a In f t B  (%e) as = O  The first order of differential log likelihood function with respect to 8 is called the maximum likelihood (ML) estimate. If the efficient estimate does not exist, then the ML estimate will not achieve the lower bound and hence it is difficult to ascertain how closely the variance of any estimate will approach the bound. Figure 2 show that the LLF simulation results for the alpha-stable with a = (0 ,451. x =[-6. -61, with y =  0.1. Its 2" order LLF distributions are shown in Figure 3, which shows the view from the a axis of Figure 2, from which we can see why generally the value of a h a s  been taken in the range [1,21. The value about 1.5 is strongly recommended if there is no information about a. Figure 2:  The 2" order simulations of the LLF for an alpha-stable with g = 0.1 Figure 3: The view of the results of Figure 2 from the front of the alpha-axis. 3. BAYESIAN ESTIMATOR If we take the probability density of Bas p ( @ :  and the posterior density function as f(@ I X, ,..., X,) , then the updated orobability densilv function of Bis as follows: f (e ,x ,  ...., x,) f (X,  ..., X") f(B I x,, ..., X") = - (6) p(e)  f(x ,,..., X, I e) j f ( ~ ~ , . . . . ~ ~  ie)p(e)de If we estimate the parameters of the prior distributions of the signal sand noise q components of the wavelet coeffcients c, we may use the parameters to form the prior PDFs of PSs) and P&), hence the inpulloutput relationship can be established by the Bayesian estimator. namely, let inpuUoutput of the Bayesian estimator =BE, we have: 71 0 (7) Figure 4 shows how yaffect the BE and Figure 5 shows how U affects the BE. (i."a- , l lDm"l l l i~ fo-" IBO,0 . ,D,  0.15,020.0*1. om, O I V l d  1 (I 30 20 a P e $ 0  2 s 3 -10 I $ $ 0  2 0  0 2033 1_ €400 B a a -  4- 12033 I" >m 40 Dll. Figure 4: How yaffects BE. Here y ranges from 0.08 to 1.0, with U= 4.5. a= 1.702. slg- -1"rphllwplot-l 1 . 0 , 2 0 , 2 1 , 1 0 , l D ,  , I O .  100. a" '1 Figure 5: How oaffects BE. Here oranges from 1.0 to IO. with y = 0.3, a= 1.702. As we mentioned above for the Bayesian estimators the three parameters are important and can affect the BE strongly. Achim et al. 181 tried to take yo as one parameter and showed a simulation hy keeping vu constant, but the fact is that even though the ratio is the same the BE can he significantly different. However, it is noted that the o i s  generally estimated and we may even have no information about it. However, as in section E we may take a= 1.5 as  fist estimate. Hence we can obtain the BE from the Figure 6. where yuis taken tiom 0.1. to 0.8. It is clearly seen that when ratio equals to 0.8 the BE becomes the similar to the case of the hard threshold presented by Donoho. It is to be noticed that the identity line can be a reference for individual BE curve. <m 1600 m 21W 3MO 31W 4000 Dab Figure 6: Individual BE with ratio of (yo) equals to 0.1.0.2,0.E5. 0.5 and 0.8 and a= 1.5 taken from section 2. The BE with 0.8 is similar to "hard threshold" case. 4. SOME EXAMPLES When we make measurements. we have no information about the noise value of the image we obtained. The only information one may have is an experience in judgement of the noise level, which becomes the outline of the denoising strategy. Figure 7 shows the denoising with Haar wavelet in 2 levels. Figure 7: Denoising image of the web page of "University of Canberra" by Haar wavelet in 2 levels. Figure 8 shows the noisy image. If we did not have any information about the noisy image, we can. as mentioned above, take a= 1.5 and (Yo) = 0.5 and the denoised image is in Figure 9. The results by other methods, together with the Bayesian estimator for a-stable method, are given in the table 1. 71 1 stable distribution. Comparisons of other denoising results are in table 1. Method 1 1  1 2  1 3  1 4  SMSE 112.21 I 12.30 I 12.01 I 12.89 Table  1: Comparison of denoising results with BE in  signal to mean square error (S/MSE) in dB. Here I= soft thresholding: 2 = Hard thresholding; 3 = Homomorphic Wiener; 4 = Bayesian estimator based on a-stable 4. CONCLUSION A new technique for denoising an image has been developed. The technique uses the statistician's Bayesian estimator theory to simplify the selection of parameters. and in some situations it provides more precise images than other methods. 5. REFERENCES 1. D.L. Donoho and I.M. Johnstone, '"Wavelet shrinkage: Asymptopia?", Journal of the Royal Statistical Society, Series B, vol. 57. 301-369, 1995. Y. XU. J.B. Weaver. D.M. Healy. and J. Lu, "Wavelet transform domain filtration technique". IEEE Transactions on Image Processing, vol. 3. 747.758, 1994. D. L. Donoho. "Nonlinear wavelet methods for recovery of signals. densities, and spectra from indirect and noisy data". In Proceeding of Symposia in Applied Mathematic. American Mathematical Society vol. 00, 173-205. 1993. 4. D. L. Donoho, "Denoising by sofl-thresholding," IEEE Trans. Inform. Theory, vol. 41. pp613-627, May 1995. 2. 3. 5. 6. 7. 8. 9. 10 11 12 13 14 15 16 E.P. Simoncelli and E. H. Adelson. "Noise removal via Bayesian wavelet coring," Third lnt. Conf. On Image Processing, vol. 1. pp. 379-382, September 1996. X. Zong, A. F. Laine, and E. A. Geiser. "Speckle reduction and contrast enhancement of echocardiograms via multiscale nonlinear processing," IEEE Trans. Med. Imag., vol. 17, pp. 532-540. Aug. 1998. M. Popescu, P. Cristea, and A. Bezerianos, "Multiresolution distributed filtering: A novel technique that reduces the amount of data rewired in hieh resolution electrocardiography," Future Gen. Comp. Sys., no. 15, pp. 195-209. 1999. ~~ ~. ~~~~ A. Achim, A. Bezerianos. and P. Tsakalides, "An Aplha- stable based Bayesian algorithm for speckle noise removal in the wavelet domain." Proc. NSIP-01. June 03-06. 2001. Baltimore. Maryland USA. D. L. Donoho and I. M. Johnstone. "ldeal spatial adaptation by wavelet shrinkage", Biometrika. Also Tech. Repon of Statistics. July, 1992. revised April 1993. G. P. Nason, "Wavelet regression by cross-validation". Technical Repon 447. Stanford University, Statistics Department, March 1994. B. R. Hunt and 0. Kubler. "Karhunen-Loeve multispectral image restoration. Part I: theory". IEEE Trans. On Acoustic Speech and Signal. Proc., ASSP vol. 32.592-600.1984. N. P. Galatssanos and R. T. Chin, "Digital restoration of multichannel images", IEEE Trans. Acoust. SP.. and Sign. Proco., vol. 37.415-426, 1989. M. K. Ozhan, A, T. Erden, M. 1. Sezan. and A. M. Tekalp. "Efficient multiframe Wiener restoration of blurred and noisv imaee seauences". IEEE Trans. Imaee Proc.. vol. 1. 4531476, 1992. ' Eero P. Simonceili, "Baysian denoising of visual images in the wavelet domain", Bavesian Inference in Wavelet Based IModels eds., Pmuller and- B Vidakovic, Chapter 18. pp29l- 308, lecture notes in statistics, vol. 141, Springer-Verlag. New York, 1999. X. Huang and G. A. Woolsey, "Image denoising using Wiener filtering and wavelet thresholding", IEEE lntemational Conference, ICME 2000. Proc. Vol. 3 ppl759- 1762.2000. X. Huang.. "Denoising Image via Minimum Variance Bound Bavesian Estimator." IEEE International Conf. ICME2001. Proc., pp533-536. 2001. 17. E. P. Simoncell and E. H. Adelson. "Noise removal via Bayesian wavelet coring". In Third Int. Conf. On Image Proc., vol. 1. pp379-382, Lausanne, September 1996. IEEE Sig. Proc. Society. 18. S. G. Mallat, "A theory for multiresolution signal decomposition: the wavelet representation," IEEE Trans. Pattern Anal. Machine Intell. Vol. 11. pp-674- ~~ 692, July 1989. 19. P. Tsakalides. P. Reveliotis. and C. L. Nikias. "Scalar quantization of heavy-tailed signals," IEE Proc: Vision, hag .  Sign. Proc.. vol. 147. pp. 475-484. Oct. znnn. 20. S. R. Peterson, Y. H. Lee, S. A. Kassam. "Some statistical properties of alpha-trimmed mean and standard type M filten." IEEE on Acoust., Speech, Sig. Proc.. vol. 36,110.5, 00707-713, 1988. 71 2 

Maximum Likelihood for Bayesian Estimator Based on alpha-Stable For Image

https://researchsystem.canberra.edu.au/ws/files/29609712/01035880.pdf

Maximum Likelihood for Bayesian Estimator Based on alpha-Stable For Image

Abstract

Similar works

Full text

Available Versions

University of Canberra Research Repository