Statistical algorithm for nonuniformity correction in focal-plane arrays

A statistical algorithm has been developed to compensate for the fixed-pattern noise associated with spatial nonuniformity and temporal drift in the response of focal-plane array infrared imaging systems. The algorithm uses initial scene data to generate initial estimates of the gain, the offset, and the variance of the additive electronic noise of each detector element. The algorithm then updates these parameters by use of subsequent frames and uses the updated parameters to restore the true image by use of a least-mean-square error finite-impulse-response filter. The algorithm is applied to infrared data, and the restored images compare favorably with those restored by use of a multiple-point calibration technique

Scene-Based Nonuniformity Correction with Reduced Ghosting Using a Gated LMS Algorithm

In this paper, we present a scene-based nouniformity correction (NUC) method using a modified adaptive least mean square (LMS) algorithm with a novel gating operation on the updates. The gating is designed to significantly reduce ghosting artifacts produced by many scene-based NUC algorithms by halting updates when temporal variation is lacking. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods including other LMS and constant statistics based methods. The experimental results include simulated imagery and a real infrared image sequence. We show that the proposed method significantly reduces ghosting artifacts, but has a slightly longer convergence time

Multi-Model Kalman Filtering for Adaptive Nonuniformity: Correction in Infrared Sensors

This paper presents an adaptive technique for the estimation of nonuniformity parameters of infrared focal-plane arrays that is robust with respect to changes and uncertainties in scene and sensor characteristics. The proposed algorithm is based on using a bank of Kalman filters in parallel. Each filter independently estimates state variables comprising the gain and the bias matrices of the sensor, according to its own dynamical-model parameters, which underly the statistics of the scene and the nonuniformity as well as the temporal drift in the nonuniformity. The supervising component of the algorithm then generates the final estimates of the state variables by forming a weighted superposition of all the estimates rendered by each Kalman filter. The weights are obtained according to the a posteriori -likelihood principle, applied to the family of models by considering the output residual errors associated with each filter. These weights are updated iteratively between blocks of data, providing the estimator the means to follow the dynamics of the scenes and the sensor. The performance of the proposed estimator and its ability to compensate for fixed-pattern noise are tested using both real and simulated data. The real data is obtained using two cameras operating in the mid- and long-wave infrared regime

Simultaneous temperature estimation and nonuniformity correction from multiple frames

Infrared (IR) cameras are widely used for temperature measurements in various applications, including agriculture, medicine, and security. Low-cost IR camera have an immense potential to replace expansive radiometric cameras in these applications, however low-cost microbolometer-based IR cameras are prone to spatially-variant nonuniformity and to drift in temperature measurements, which limits their usability in practical scenarios. To address these limitations, we propose a novel approach for simultaneous temperature estimation and nonuniformity correction from multiple frames captured by low-cost microbolometer-based IR cameras. We leverage the physical image acquisition model of the camera and incorporate it into a deep learning architecture called kernel estimation networks (KPN), which enables us to combine multiple frames despite imperfect registration between them. We also propose a novel offset block that incorporates the ambient temperature into the model and enables us to estimate the offset of the camera, which is a key factor in temperature estimation. Our findings demonstrate that the number of frames has a significant impact on the accuracy of temperature estimation and nonuniformity correction. Moreover, our approach achieves a significant improvement in performance compared to vanilla KPN, thanks to the offset block. The method was tested on real data collected by a low-cost IR camera mounted on a UAV, showing only a small average error of $0.27^\circ C-0.54^\circ C$ relative to costly scientific-grade radiometric cameras. Our method provides an accurate and efficient solution for simultaneous temperature estimation and nonuniformity correction, which has important implications for a wide range of practical applications

Projection-based image registration in the presence of fixed-pattern noise

A computationally efficient method for image registration is investigated that can achieve an improved performance over the traditional two-dimensional (2-D) cross-correlation-based techniques in the presence of both fixed-pattern and temporal noise. The method relies on transforming each image in the sequence of frames into two vector projections formed by accumulating pixel values along the rows and columns of the image. The vector projections corresponding to successive frames are in turn used to estimate the individual horizontal and vertical components of the shift by means of a one-dimensional (1-D) cross-correlation-based estimator. While gradient-based shift estimation techniques are computationally efficient, they often exhibit degraded performance under noisy conditions in comparison to cross-correlators due to the fact that the gradient operation amplifies noise. The projection-based estimator, on the other hand, significantly reduces the computational complexity associated with the 2-D operations involved in traditional correlation-based shift estimators while improving the performance in the presence of temporal and spatial noise. To show the noise rejection capability of the projection-based shift estimator relative to the 2-D cross correlator, a figure-of-merit is developed and computed reflecting the signal-to-noise ratio (SNR) associated with each estimator. The two methods are also compared by means of computer simulation and tests using real image sequences

Static Scene Statistical Non-Uniformity Correction

Non-Uniformity Correction (NUC) is required to normalize imaging detector Focal-Plane Array (FPA) outputs due to differences in the end-to-end photoelectric responses between pixels. Currently, multi-point NUC methods require static, uniform target scenes of a known intensity for calibration. Conversely, scene-based NUC methods do not require a priori knowledge of the target but the target scene must be dynamic. The new Static Scene Statistical Non-Uniformity Correction (S3NUC) algorithm was developed to address an application gap left by current NUC methods. S3NUC requires the use of two data sets of a static scene at different mean intensities but does not require a priori knowledge of the target. The S3NUC algorithm exploits the random noise in output data utilizing higher order statistical moments to extract and correct fixed pattern, systematic errors. The algorithm was tested in simulation and with measured data and the results indicate that the S3NUC algorithm is an accurate method of applying NUC. The algorithm was also able to track global array response changes over time in simulated and measured data. The results show that the variation tracking algorithm can be used to predict global changes in systems with known variation issues

Static Scene Statistical Non-Uniformity Correction

Non-Uniformity Correction (NUC) is required to normalize imaging detector Focal-Plane Array (FPA) outputs due to differences in the end-to-end photoelectric responses between pixels. Currently, multi-point NUC methods require static, uniform target scenes of a known intensity for calibration. Conversely, scene-based NUC methods do not require a priori knowledge of the target but the target scene must be dynamic. The new Static Scene Statistical Non-Uniformity Correction (S3NUC) algorithm was developed to address an application gap left by current NUC methods. S3NUC requires the use of two data sets of a static scene at different mean intensities but does not require a priori knowledge of the target. The S3NUC algorithm exploits the random noise in output data utilizing higher order statistical moments to extract and correct fixed pattern, systematic errors. The algorithm was tested in simulation and with measured data and the results indicate that the S3NUC algorithm is an accurate method of applying NUC. The algorithm was also able to track global array response changes over time in simulated and measured data. The results show that the variation tracking algorithm can be used to predict global changes in systems with known variation issues

IEEE TRANSACTIONS ON IMAGE PROCESSING, 2014 (preprint) 1 Joint Removal of Random and Fixed-Pattern Noise through Spatiotemporal Video Filtering

Abstract—We propose a framework for the denoising of videos jointly corrupted by spatially correlated (i.e. non-white) random noise and spatially correlated fixed-pattern noise. Our approach is based on motion-compensated 3-D spatiotemporal volumes, i.e. a sequence of 2-D square patches extracted along the motion trajectories of the noisy video. First, the spatial and temporal correlations within each volume are leveraged to sparsify the data in 3-D spatiotemporal transform domain, and then the coefficients of the 3-D volume spectrum are shrunk using an adaptive 3-D threshold array. Such array depends on the particular motion trajectory of the volume, the individual power spectral densities of the random and fixed-pattern noise, and also the noise variances which are adaptively estimated in transform domain. Experimental results on both synthetically corrupted data and real infrared videos demonstrate a superior suppression of the random and fixed-pattern noise from both an objective and a subjective point of view. Index Terms—Video denoising, spatiotemporal filtering, fixedpattern noise, power spectral density, adaptive transforms, thermal imaging. I

Data compressive paradigm for spectral sensing and classification using electrically tunable detectors

This dissertation contains three major parts: (1) demonstration of the algorithmic spectrometry in the mid-IR sensing regime using spectrally tunable quantum dots-in-a-well (DWELL) IR detector without employing any spectral filters; (2) further demonstration of the spectral-classification capability of tunable DWELL IR focal-plane array (FPA), again without using any spectral filters; and (3) development of a generalized filter-free data-compressive spectral sensing paradigm using the DWELL detector that enables arbitrarily specified MS sensing (e.g., spectral matched filtering, slope sensing, multicolor sensing, etc.) without using any spectral filters and possibly under constrained acquisition times