In this paper, we compare the performance of multiple turbulence mitigation algorithms to restore imagery degraded by atmospheric turbulence and camera noise. In order to quantify and compare algorithm performance, imaging scenes were simulated by applying noise and varying levels of turbulence. For the simulation, a Monte-Carlo wave optics approach is used to simulate the spatially and temporally varying turbulence in an image sequence. A Poisson-Gaussian noise mixture model is then used to add noise to the observed turbulence image set. These degraded image sets are processed with three separate restoration algorithms: Lucky Look imaging, bispectral speckle imaging, and a block matching method with restoration filter. These algorithms were chosen because they incorporate different approaches and processing techniques. The results quantitatively show how well the algorithms are able to restore the simulated degraded imagery

Dapore, Alexander J.

Hardie, Russell C.

Rucci, Michael Armand

English

University of Dayton

University of DaytoneCommonsElectrical and Computer Engineering FacultyPublicationsDepartment of Electrical and ComputerEngineering5-2017Comparing Multiple Turbulence RestorationAlgorithms Performance on Noisy AnisoplanaticImageryMichael Armand RucciAir Force Research LaboratoryRussell C. HardieUniversity of Dayton, rhardie1@udayton.eduAlexander J. DaporeL-3 Communications Cincinnati ElectronicsFollow this and additional works at: http://ecommons.udayton.edu/ece_fac_pubPart of the Electromagnetics and Photonics Commons, Optics Commons, and the OtherElectrical and Computer Engineering CommonsThis Conference Paper is brought to you for free and open access by the Department of Electrical and Computer Engineering at eCommons. It hasbeen accepted for inclusion in Electrical and Computer Engineering Faculty Publications by an authorized administrator of eCommons. For moreinformation, please contact frice1@udayton.edu, mschlangen1@udayton.edu.eCommons CitationRucci, Michael Armand; Hardie, Russell C.; and Dapore, Alexander J., "Comparing Multiple Turbulence Restoration AlgorithmsPerformance on Noisy Anisoplanatic Imagery" (2017). Electrical and Computer Engineering Faculty Publications. 411.http://ecommons.udayton.edu/ece_fac_pub/411  Comparing Multiple Turbulence Restoration Algorithms Performance on Noisy Anisoplanatic Imagery Michael A. Rucci*a, Russell C. Hardieb, Alexander J. Daporec aAir Force Research Laboratory, 2241 Avionics Circle, Wright Patterson AFB, OH 45433;  bDepartment of Electrical and Computer Engineering, University of Dayton, 300 College Park, Dayton, Ohio 45459; cL3 Technologies Cincinnati Electronics, 7500 Innovation Way, Mason, Ohio 45040 ABSTRACT  In this paper, we compare the performance of multiple turbulence mitigation algorithms to restore imagery degraded by atmospheric turbulence and camera noise. In order to quantify and compare algorithm performance, imaging scenes were simulated by applying noise and varying levels of turbulence. For the simulation, a Monte-Carlo wave optics approach is used to simulate the spatially and temporally varying turbulence in an image sequence. A Poisson-Gaussian noise mixture model is then used to add noise to the observed turbulence image set. These degraded image sets are processed with three separate restoration algorithms: Lucky Look imaging, bispectral speckle imaging, and a block matching method with restoration filter. These algorithms were chosen because they incorporate different approaches and processing techniques. The results quantitatively show how well the algorithms are able to restore the simulated degraded imagery. Keywords: Imaging, turbulence, anisoplanatism, Monte-Carlo, wave optics, noise, restoration  1. INTRODUCTION An imaging system is susceptible to a wide range of degradations that limit its performance. In situations where data is captured over long distances, such as in the astronomical case, the limiting factor in the camera’s performance is typically due to turbulence.  Turbulence artifacts are due to continual changes in temperature and pressure that lead to random fluctuations in the index of refraction1. These continual changes in the atmosphere lead to warping and blurring in collected imagery. The strength of turbulence can be quantified by a Cn2 profile along the path. Additionally this profile is used to calculate the Fried parameter1,2 and the isoplanatic angle1. Both of these products convey information on how an atmospheric point spread function (PSF) will behave. In cases where the FOV subtends many isoplanatic patches, the turbulent short exposure PSF’s now become space variant. This in turn leads to a more challenging case for turbulence mitigation. It is important to note that in either imaging scenario the Fried parameter used for the long exposure case PSF is space invariant.  It is well established there is continual research to simulate and mitigate turbulence. Approaches to simulate turbulence3,4 allow for truth data to be available to be used in quantifying how well a mitigation approach works on varying levels of turbulence strength. Mitigation methods include, but are not limited to, multi-frame approaches and deblurring5, Lucky Look6, or speckle imaging7,8. All three methods have different variations in the approach to remove turbulence.  Another consideration in image quality is the signal to noise ratio (SNR). A higher SNR level allows for more contrast in the imagery. However, there are cases where a short integration time is used with the camera to freeze the effects of turbulence. In doing so the signal levels of the imagery remain lower than a long exposure case.  Typically, just as with trying to model and mitigate turbulence, a camera’s noise statistics are modeled with an additive Gaussian noise term9,10. A more accurate representation of the total camera system noise11,12 is through a Poisson-Gaussian noise mixture model. In this paper we combine simulated turbulence with a Poisson-Gaussian noise model to survey how well the three main outlined algorithms perform in restoring the degraded data. Section 2 discusses the modeling/simulation approach. The results of the algorithms are presented in Section 3. Finally, a conclusion is provided in Section 4. Long-Range Imaging II, edited by Eric J. Kelmelis, Proc. of SPIE Vol. 10204, 1020409 · © 2017 SPIE · CCC code: 0277-786X/17/$18 · doi: 10.1117/12.2269133Proc. of SPIE Vol. 10204  1020409-1Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx  2. OBSERVATION MODEL The forward model used to describe noisy turbulent data stems from previous work4,13. In these papers the ݊th observed image is a culmination of warping and blurring, such that  ݃(ݔ, ݕ)௡ = ݏ(ݔ, ݕ)௡ሼℎ(ݔ, ݕ)௔௡ሾ݂(ݔ, ݕ)ሿሽ +  ߟ(ݔ, ݕ)௡. (1) The ideal frame is ݂(ݔ, ݕ) with ݔ, ݕ being spatial coordinates. A position-dependent shift operator, ݏ(ݔ, ݕ)௡(), describes the warping seen in the imagery. In turn, the position-dependent/position corrected blurring is included in ℎ(ݔ, ݕ)௔௡(). Note the above model only includes additive iid Gaussian noise, ߟ(ݔ, ݕ)௡, that has a zero mean with a known variance.  As discussed earlier in and in other papers11,12, a Poisson-Gaussian noise model is more accurate. Thus, part of the equation above can be rewritten for simplicity as ݖ(ݔ, ݕ)௡ = ݏ(ݔ, ݕ)௡ሼℎ(ݔ, ݕ)௔௡ሾ݂(ݔ, ݕ)ሿሽ .                                                                   (2)     The warped/blurred image, ݖ(ݔ, ݕ)௡, is then added into the Possion-Gaussian noise model to represent the observed frame as   ݃(ݔ, ݕ)௡ = ߙܲ(ߙିଵݖ(ݔ, ݕ)௡) +  ߟ(ݔ, ݕ)௡.       (3) In the equation above ܲ is a Poisson random process contributing the signal dependent component of the noise and ߙ is a scalar value representing the camera gain. For the purposes of generic noise modeling, versus a physical cameras performance, we will allow ߙ to be one for the simulations in this paper. In the revised noise mode we have neglected to show a camera offset term as it is typically zero11. The camera model used in the turbulence simulations is the same as in a previous paper4, and shown again for convenience in Table 1. Table 1. Information on the camera system used in the simulations. Parameter Value Aperture D = 0.2034m Focal length l = 1.2m F-number f/# =5.9 Wavelength λ = 0.525µm Object distance L = 7km Nyquist pixel spacing (focal plane) δf = 1.5488µm Nyquist pixel spacing (object plane) δo = 9.0344mm Input image dynamic range 256 digital units Additive Gaussian noise variance 4 digital units  The camera parameters listed above were chosen to represent a generic imaging system. The wavelength of the system certainly allows for turbulent imagery to be collected over the horizontal slant range of seven kilometers. A input dynamic range of 8 bits was chosen due to the fact the camera would be collecting short exposure imagery to freeze the turbulence. The f-number of the system was also chosen to ensure the system was limited by the turbulent component of the modulation transfer function (MTF). The turbulence parameters also follow from a previous paper4 and are given in Table 2. The algorithm used was a Monte-Carlo wave optics approach using a set number of equally spaced phase screens between the object and the camera. Any aspects of camera motion were not incorporated into the simulation data sets generated for this survey. Instead, the camera and object were modeled as remaining stationary during the time of the collection.  Proc. of SPIE Vol. 10204  1020409-2Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx  Table 2. Information on the turbulence parameters used in the simulations. Parameter Value Path length L = 7km Propagation step Δz = 700m Cropped screen samples ࣨ = 256 Propagation screen width X = 0.9699 m Pupil plane point spread ܦ෩= 0.8136 m Propagation sample spacing Δx = 0.0038m Number of phase screens  N = 10 (9 non-zero) Phase screen type Modified von Kܽ́rmܽ́n with subharmonics Inner scale lo= 0.01m Outer scale Lo=300m Image size (pixels) 257×257 Image size (object plane) 2.3218×2.3218 m  Pixel skip 4 pixels (65×65 PSF array)  These turbulence parameters were used to simulate turbulence over a horizontal path where the Cn2 would be a constant value. All turbulent datasets used in this survey were generated by a method discussed in another paper4 and also used in a restoration paper5. The restoration paper however did not use a Poisson-Gaussian noise mixture model. These two papers provide greater details on the specific details used to create the simulated turbulence products. Camera noise was then added after the turbulence was simulated following the equations outlined above. 3. RESULTS In this section we present a variety of restoration results. Three main algorithms were chosen to process four levels of turbulence. The first algorithm is the block-matching Wiener filter (BMWF) turbulence mitigation system5. The main parameters used in the BMWF algorithm is the block matching algorithm (BMA) that had the window size set to 15 ×15 pixels. The other component is the noise to signal ratio (NSR) used in the Wiener filter portion of the algorithm. The algorithm was able to choose the best NSR value to minimize the mean absolute error (MAE) as the true object scene is known. The Wiener filter utilized the point spread function (PSF) of the imaging system outlined in Table 1, along with the estimated atmospheric coherence diameter determined from the reduction in warping during image restoration as discussed in the previous paper.  The second algorithm used in this evaluation is the Sobolev gradient flow (SOB) plus Laplacian (LAP) referred to in the literature as SOB+LAP14. This algorithm mitigates turbulence in two parts by using a Laplacian operator to mitigate temporal distortions in the imagery. The second part involves using a Sobolev gradient method to sharpen the content in the individual frames. The MATLAB code for the SOB+LAP algorithm is available online15. The SOB+LAP & Lucky Look algorithm does require a degraded video sequence and outputs a restored sequence. This capability is very useful for applications when a restored video sequence is required over the whole input sequence. The output video sequence can be temporally averaged to produce a single frame result. The SOB+LAP output data sequence could also be further processed with a Lucky Look fusion6 approach as done in this paper. The Lucky Look fusion Gaussian blur parameter was optimized to yield the best MAE. Proc. of SPIE Vol. 10204  1020409-3Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx   The third algin a previousperformanceand 100 framFigure 1. observed and (d) thIn the imagerFig.1 shows 2/3. The BMWLook case thpresent in Fismall residuaFig.1 showcaoverview of temporal avehigher frequeresult and trbetween the number of frto distinguishorithm used to paper5 the ap. The NSR wae sequence ovComparison of noisy turbulent e bispectral spey above it is cthe final outpuF algorithm be noise is sigg. 1(b). In thel amount of noses one specifthe results is sraging that thency content thuth image wilBMWF and tames increase  for the SOB+ process the nodization uses once again oer 4 levels of tprocessed noisyframe, (b) the Bckle imaging melear to see in Ft frame after palances the nonificantly redu bispectral spise in the imaic data point whown in Fig.2 SNR will incey could potel be higher thhe SOB+LAP for just one tuLAP & Luckyoisy turbulent d 16×16 pixelptimized to prurbulence stre turbulent data oMWF turbulencthod. ig. 1(a) the imrocessing 100ise suppressioced. The imaeckle imaginggery. ith varying lev. As the numbrease. As algontially not usean an algorith& Lucky Loorbulence case.  Look algorithdata is the bisp tiles8. These oduce the smangth. ver a 100 framee mitigation syspact from both frames of noisn along with dge though app case the imaels of turbulener of frames inrithms try to m all the availam that utilizek. The SOB+The differencm as the turbuectral speckletiles were alsollest MAE. Th sequence whertem, (c) the SOB turbulence any turbulent daeblurring the ears to have lgery retains hce with and wcrease it is exake smarter dble frames. In s the informaLAP & Luckye between the lence increase imaging meth locally registe algorithms pe the Cn2 = 1.0 ×+LAP with Lud noise.  The ta with a consimagery. In thess of the higigher spatial fithout noise. Apected in the ecisions abouturn, the MAEtion in all the Look holds clean and noiss. od7,8,16. Once ered to providrocessed a 25  10-16 m-2/3. (a) cky Look fusionsubsequent imtant Cn2 = 1.0×e SOB+LAP &her frequencyrequencies. Th more compregeneric case ot what patches between the  frames. Thisthe same MAy case becomeagain as e better , 50, 75, an , agery in 10-16 m- Lucky  content ere is a hensive f simple  contain restored  is case E as the s harder Proc. of SPIE Vol. 10204  1020409-4Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx  Figure 2. The BMWF are more framalso holds a portions of thnot available In this papermitigation syhelped to addfreeze the tuimaging sceturbulence stimagery yieldOverview of thealgorithm shoes. Noise in tsteady MAE we algorithms  to help optimi we have exastem, SOB+L appropriate nrbulence resulnario. Each arength datasetss the best resu algorithms perws consistent he imagery onith and withoused tuned NSze the algorithmined how wAP & Lucky oise to the shots in lower SNlgorithm prov. The survey ilts. formance with vperformance. ly causes a sliut noise overR values to prm. 4. CONell noisy aniLook, and a brt exposure tuR imagery. Tides differentndicates that uarying levels ofThe turbulenceght increase in varying leveloduce the besCLUSIONsoplanatic turbispectral specrbulence simuhe joint turbu MAE trendstilizing all the input frames an only case ha the MAE. Fis of turbulenct MAE value.  ulent data is kle algorithm.lated data setslence and low throughout t information ind turbulence levs a slowly decnally, the bispe. It is importaIn real applicarestored by t The Poisson-. A short integ SNR datasethe various nu the time domels.  reasing MAE ectral speckle nt to remembtions the true he BMWF turGaussian noisration time reqs provide an mber of framain for short e as there imaging er some scene is bulence e model uired to accurate es and xposure Proc. of SPIE Vol. 10204  1020409-5Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx  ACKNOWLEDGMENTS The work used in this paper has been supported in part by funding from L-3 Communications Cincinnati Electronics and under AFRL Award Nos. FA8650-10-2-7028 and FA9550-14-1-0244. REFERENCES [1] Michael C Roggemann, Byron M. Welsh, and Bobby R. Hunt. [Imaging through turbulence]. CRC press, (1996). [2] David L. Fried, "Limiting resolution looking down through the atmosphere." JOSA 56.10, 1380-1384 (1966) [3] Svetlana L. Lachinova, et al. "Anisoplanatic imaging through atmospheric turbulence: Brightness function approach." Optical Engineering+ Applications. International Society for Optics and Photonics, (2007) [4] Russell C Hardie, et al. "Simulation of anisoplanatic imaging through optical turbulence using numerical wave propagation with new validation analysis." Optical Engineering 56.7, 071502-071502 (2017) [5] Russell C Hardie, et al "Block matching and Wiener filtering approach to optical turbulence mitigation and its application to simulated and real imagery with quantitative error analysis." Optical Engineering 56.7,  071503-071503 (2017) [6] Mathieu Aubailly, et al. "Automated video enhancement from a stream of atmospherically-distorted images: the lucky-region fusion approach." SPIE Optical Engineering+ Applications. International Society for Optics and Photonics, (2009) [7] Taylor W Lawrence, et al. "Extended-image reconstruction through horizontal path turbulence using bispectral speckle interferometry." Optical Engineering 31.3, 627-636 (1992) [8] Carmen J Carrano, "Speckle imaging over horizontal paths." International Symposium on Optical Science and Technology. International Society for Optics and Photonics, (2002) [9] Dalong Li, Russell M. Mersereau, and Steven Simske. "Atmospheric turbulence-degraded image restoration using principal components analysis." IEEE Geoscience and Remote Sensing Letters 4.3, 340-344 (2007) [10] Luxin Yan, et al. "Atmospheric-turbulence-degraded astronomical image restoration by minimizing second-order central moment." IEEE Geoscience and Remote Sensing Letters 9.4, 672-676 (2012) [11] Michael Rucci, Russell C. Hardie, and Kenneth J. Barnard. "Computationally efficient video restoration for Nyquist sampled imaging sensors combining an affine-motion-based temporal Kalman filter and adaptive Wiener filter." Applied optics 53.13, C1-C13. (2014) [12] Markku Mäkitalo, and Alessandro Foi. "Poisson-gaussian denoising using the exact unbiased inverse of the generalized anscombe transformation." Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on. IEEE, (2012) [13] Donald Fraser, Glen Thorpe, and Andrew Lambert. "Atmospheric turbulence visualization with wide-area motion-blur restoration." JOSA A 16.7, 1751-1758 (1999) [14] Y Lou, S H Kang, S Soatto, and A L Bertozzi. Video stabilization of atmospheric turbulence distortion. Inverse Problems in Imaging, Special Issue in honor of Tony Chan,7(3), 839 - 861 (2013) [15] https://sites.google.com/site/louyifei/research/turbulence [16] Glen E Archer, Jeremy P. Bos, and Michael C. Roggemann. "Comparison of bispectrum, multiframe blind deconvolution and hybrid bispectrum-multiframe blind deconvolution image reconstruction techniques for anisoplanatic, long horizontal-path imaging." Optical Engineering 53.4, 043109-043109 (2014) Proc. of SPIE Vol. 10204  1020409-6Downloaded From: http://proceedings.spiedigitallibrary.org/ on 06/23/2017 Terms of Use: http://spiedigitallibrary.org/ss/termsofuse.aspx

Comparing Multiple Turbulence Restoration Algorithms Performance on Noisy Anisoplanatic Imagery

https://ecommons.udayton.edu/cgi/viewcontent.cgi?article=1408&context=ece_fac_pub

Comparing Multiple Turbulence Restoration Algorithms Performance on Noisy Anisoplanatic Imagery

Abstract

Similar works

Full text

Available Versions

University of Dayton