Scene-based nonuniformity correction with video sequences and registration

We describe a new, to our knowledge, scene-based nonuniformity correction algorithm for array detectors. The algorithm relies on the ability to register a sequence of observed frames in the presence of the fixed-pattern noise caused by pixel-to-pixel nonuniformity. In low-to-moderate levels of nonuniformity, sufficiently accurate registration may be possible with standard scene-based registration techniques. If the registration is accurate, and motion exists between the frames, then groups of independent detectors can be identified that observe the same irradiance (or true scene value). These detector outputs are averaged to generate estimates of the true scene values. With these scene estimates, and the corresponding observed values through a given detector, a curve-fitting procedure is used to estimate the individual detector response parameters. These can then be used to correct for detector nonuniformity. The strength of the algorithm lies in its simplicity and low computational complexity. Experimental results, to illustrate the performance of the algorithm, include the use of visible-range imagery with simulated nonuniformity and infrared imagery with real nonuniformity

Statistical algorithm for nonuniformity correction in focal-plane arrays

A statistical algorithm has been developed to compensate for the fixed-pattern noise associated with spatial nonuniformity and temporal drift in the response of focal-plane array infrared imaging systems. The algorithm uses initial scene data to generate initial estimates of the gain, the offset, and the variance of the additive electronic noise of each detector element. The algorithm then updates these parameters by use of subsequent frames and uses the updated parameters to restore the true image by use of a least-mean-square error finite-impulse-response filter. The algorithm is applied to infrared data, and the restored images compare favorably with those restored by use of a multiple-point calibration technique

Simultaneous temperature estimation and nonuniformity correction from multiple frames

Infrared (IR) cameras are widely used for temperature measurements in various applications, including agriculture, medicine, and security. Low-cost IR camera have an immense potential to replace expansive radiometric cameras in these applications, however low-cost microbolometer-based IR cameras are prone to spatially-variant nonuniformity and to drift in temperature measurements, which limits their usability in practical scenarios. To address these limitations, we propose a novel approach for simultaneous temperature estimation and nonuniformity correction from multiple frames captured by low-cost microbolometer-based IR cameras. We leverage the physical image acquisition model of the camera and incorporate it into a deep learning architecture called kernel estimation networks (KPN), which enables us to combine multiple frames despite imperfect registration between them. We also propose a novel offset block that incorporates the ambient temperature into the model and enables us to estimate the offset of the camera, which is a key factor in temperature estimation. Our findings demonstrate that the number of frames has a significant impact on the accuracy of temperature estimation and nonuniformity correction. Moreover, our approach achieves a significant improvement in performance compared to vanilla KPN, thanks to the offset block. The method was tested on real data collected by a low-cost IR camera mounted on a UAV, showing only a small average error of $0.27^\circ C-0.54^\circ C$ relative to costly scientific-grade radiometric cameras. Our method provides an accurate and efficient solution for simultaneous temperature estimation and nonuniformity correction, which has important implications for a wide range of practical applications

Shift Estimation Algorithm for Dynamic Sensors With Frame-to-Frame Variation in Their Spectral Response

This study is motivated by the emergence of a new class of tunable infrared spectral-imaging sensors that offer the ability to dynamically vary the sensor\u27s intrinsic spectral response from frame to frame in an electronically controlled fashion. A manifestation of this is when a sequence of dissimilar spectral responses is periodically realized, whereby in every period of acquired imagery, each frame is associated with a distinct spectral band. Traditional scene-based global shift estimation algorithms are not applicable to such spectrally heterogeneous video sequences, as a pixel value may change from frame to frame as a result of both global motion and varying spectral response. In this paper, a novel algorithm is proposed and examined to fuse a series of coarse global shift estimates between periodically sampled pairs of nonadjacent frames to estimate motion between consecutive frames; each pair corresponds to two nonadjacent frames of the same spectral band. The proposed algorithm outperforms three alternative methods, with the average error being one half of that obtained by using an equal weights version of the proposed algorithm, one-fourth of that obtained by using a simple linear interpolation method, and one-twentieth of that obtained by using a nai¿ve correlation-based direct method

Projection-based image registration in the presence of fixed-pattern noise

A computationally efficient method for image registration is investigated that can achieve an improved performance over the traditional two-dimensional (2-D) cross-correlation-based techniques in the presence of both fixed-pattern and temporal noise. The method relies on transforming each image in the sequence of frames into two vector projections formed by accumulating pixel values along the rows and columns of the image. The vector projections corresponding to successive frames are in turn used to estimate the individual horizontal and vertical components of the shift by means of a one-dimensional (1-D) cross-correlation-based estimator. While gradient-based shift estimation techniques are computationally efficient, they often exhibit degraded performance under noisy conditions in comparison to cross-correlators due to the fact that the gradient operation amplifies noise. The projection-based estimator, on the other hand, significantly reduces the computational complexity associated with the 2-D operations involved in traditional correlation-based shift estimators while improving the performance in the presence of temporal and spatial noise. To show the noise rejection capability of the projection-based shift estimator relative to the 2-D cross correlator, a figure-of-merit is developed and computed reflecting the signal-to-noise ratio (SNR) associated with each estimator. The two methods are also compared by means of computer simulation and tests using real image sequences

Scene-Based Nonuniformity Correction with Reduced Ghosting Using a Gated LMS Algorithm

In this paper, we present a scene-based nouniformity correction (NUC) method using a modified adaptive least mean square (LMS) algorithm with a novel gating operation on the updates. The gating is designed to significantly reduce ghosting artifacts produced by many scene-based NUC algorithms by halting updates when temporal variation is lacking. We define the algorithm and present a number of experimental results to demonstrate the efficacy of the proposed method in comparison to several previously published methods including other LMS and constant statistics based methods. The experimental results include simulated imagery and a real infrared image sequence. We show that the proposed method significantly reduces ghosting artifacts, but has a slightly longer convergence time

Intensity Nonuniformity Correction for Brain MR Images with Known Voxel Classes

Intensity nonuniformity in magnetic resonance (MR) images, represented by a smooth and slowly varying function, is a typical artifact that is a nuisance for many image processing methods. To eliminate the artifact, we have to estimate the nonuniformity as a smooth and slowly varying function and factor it out from the given data. We reformulate the problem as a problem of finding a unique smooth function in a particular set of piecewise smooth functions and propose a variational method for finding it. We deliver the main idea using a simple one-dimensional example first and provide a thorough analysis of the problem in a three-phase scenario in three dimensions whose application can be found in the brain MR images. Experiments with synthetic and real MR images and a comparison with a state-of-the-art method, N3, show our algorithm???s satisfactory performance in estimating the nonuniformity with and without noise. An automated procedure is also proposed for practical use.open

Scene-based nonuniformity correction for focal plane arrays by the method of the inverse covariance form

What is to our knowledge a new scene-based algorithm for nonuniformity correction in infrared focal-plane array sensors has been developed. The technique is based on the inverse covariance form of the Kalman filter (KF), which has been reported previously and used in estimating the gain and bias of each detector in the array from scene data. The gain and the bias of each detector in the focal-plane array are assumed constant within a given sequence of frames, corresponding to a certain time and operational conditions, but they are allowed to randomly drift from one sequence to another following a discrete-time Gauss-Markov process. The inverse covariance form filter estimates the gain and the bias of each detector in the focal-plane array and optimally updates them as they drift in time. The estimation is performed with considerably higher computational efficiency than the equivalent KF. The ability of the algorithm in compensating for fixed-pattern noise in infrared imagery and in reducing the computational complexity is demonstrated by use of both simulated and real data

A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correct

During digital video acquisition, imagery may be degraded by a number of phenomena including undersampling, blur, and noise. Many systems, particularly those containing infrared focal plane array (FPA) sensors, are also subject to detector nonuniformity. Nonuniformity, or fixed pattern noise, results from nonuniform responsivity of the photodetectors that make up the FPA. Here we propose a maximuma posteriori (MAP) estimation framework for simultaneously addressing undersampling, linear blur, additive noise, and bias nonuniformity. In particular, we jointly estimate a superresolution (SR) image and detector bias nonuniformity parameters from a sequence of observed frames. This algorithm can be applied to video in a variety of ways including using amoving temporal window of frames to process successive groups of frames. By combining SR and nonuniformity correction (NUC) in this fashion, we demonstrate that superior results are possible compared with the more conventional approach of performing scene-based NUC followed by independent SR. The proposed MAP algorithm can be applied with or without SR, depending on the application and computational resources available. Even without SR, we believe that the proposed algorithm represents a novel and promising scene-based NUC technique. We present a number of experimental results to demonstrate the efficacy of the proposed algorithm. These include simulated imagery for quantitative analysis and real infrared video for qualitative analysis