Real Time Non uniformity Correction Algorithm and Implementation in Reconfigurable Architecture for Infra red Imaging Systems

 In modern electro-optical systems, infra-red (IR) imaging system is an essential sensor used for day and night surveillance. In recent years, advancements in IR sensor technology resulted the detectors having smaller pitch, better thermal sensitivity with large format like 640.512, 1024.768 and 1280.1024. Large format IR detectors enables realisation of high resolution compact thermal imager having wide field-of view coverage. However, the performance of these infrared imaging systems gets limited by non uniformity produced by sensing element, which is temporal in nature and present in spatial domain. This non uniformity results the fixed pattern noise, which arises due to variation in gain and offset components of the each pixel of the sensor even when exposed to a uniform scene. This fixed pattern noise limits the temperature resolution capability of the IR imaging system thereby causing the degradation in system performance. Therefore, it is necessary to correct the non-uniformities in real time. In this paper, non uniformity correction algorithm and its implementation in reconfigurable architectures have been presented and results on real time data have been described

Statistical Photocalibration of Photodetectors for Radiometry without Calibrated Light Sources

Calibration of CCD arrays for identifying bad pixels and achieving nonuniformity correction is commonly accomplished using dark frames. This kind of calibration technique does not achieve radiometric calibration of the array since only the relative response of the detectors is computed. For this, a second calibration is sometimes utilized by looking at sources with known radiances. This process can be used to calibrate photodetectors as long as a calibration source is available and is well-characterized. A previous attempt at creating a procedure for calibrating a photodetector using the underlying Poisson nature of the photodetection required calculations of the skewness of the photodetector measurements. Reliance on the third moment of measurement meant that thousands of samples would be required in some cases to compute that moment. A photocalibration procedure is defined that requires only first and second moments of the measurements. The technique is applied to image data containing a known light source so that the accuracy of the technique can be surmised. It is shown that the algorithm can achieve accuracy of nearly 2.7% of the predicted number of photons using only 100 frames of image data

Scene-based nonuniformity correction for focal plane arrays by the method of the inverse covariance form

What is to our knowledge a new scene-based algorithm for nonuniformity correction in infrared focal-plane array sensors has been developed. The technique is based on the inverse covariance form of the Kalman filter (KF), which has been reported previously and used in estimating the gain and bias of each detector in the array from scene data. The gain and the bias of each detector in the focal-plane array are assumed constant within a given sequence of frames, corresponding to a certain time and operational conditions, but they are allowed to randomly drift from one sequence to another following a discrete-time Gauss-Markov process. The inverse covariance form filter estimates the gain and the bias of each detector in the focal-plane array and optimally updates them as they drift in time. The estimation is performed with considerably higher computational efficiency than the equivalent KF. The ability of the algorithm in compensating for fixed-pattern noise in infrared imagery and in reducing the computational complexity is demonstrated by use of both simulated and real data

Multi-Model Kalman Filtering for Adaptive Nonuniformity: Correction in Infrared Sensors

This paper presents an adaptive technique for the estimation of nonuniformity parameters of infrared focal-plane arrays that is robust with respect to changes and uncertainties in scene and sensor characteristics. The proposed algorithm is based on using a bank of Kalman filters in parallel. Each filter independently estimates state variables comprising the gain and the bias matrices of the sensor, according to its own dynamical-model parameters, which underly the statistics of the scene and the nonuniformity as well as the temporal drift in the nonuniformity. The supervising component of the algorithm then generates the final estimates of the state variables by forming a weighted superposition of all the estimates rendered by each Kalman filter. The weights are obtained according to the a posteriori -likelihood principle, applied to the family of models by considering the output residual errors associated with each filter. These weights are updated iteratively between blocks of data, providing the estimator the means to follow the dynamics of the scenes and the sensor. The performance of the proposed estimator and its ability to compensate for fixed-pattern noise are tested using both real and simulated data. The real data is obtained using two cameras operating in the mid- and long-wave infrared regime

IEEE TRANSACTIONS ON IMAGE PROCESSING, 2014 (preprint) 1 Joint Removal of Random and Fixed-Pattern Noise through Spatiotemporal Video Filtering

Abstract—We propose a framework for the denoising of videos jointly corrupted by spatially correlated (i.e. non-white) random noise and spatially correlated fixed-pattern noise. Our approach is based on motion-compensated 3-D spatiotemporal volumes, i.e. a sequence of 2-D square patches extracted along the motion trajectories of the noisy video. First, the spatial and temporal correlations within each volume are leveraged to sparsify the data in 3-D spatiotemporal transform domain, and then the coefficients of the 3-D volume spectrum are shrunk using an adaptive 3-D threshold array. Such array depends on the particular motion trajectory of the volume, the individual power spectral densities of the random and fixed-pattern noise, and also the noise variances which are adaptively estimated in transform domain. Experimental results on both synthetically corrupted data and real infrared videos demonstrate a superior suppression of the random and fixed-pattern noise from both an objective and a subjective point of view. Index Terms—Video denoising, spatiotemporal filtering, fixedpattern noise, power spectral density, adaptive transforms, thermal imaging. I

An accurate device for apparent emissivity characterisation in controlled atmospheric conditions up to 1423 K

Emissivity is a material property that must be measured before an accurate non-contact temperature measurement can be made. We have developed a novel instrument for measuring apparent emissivity under a controlled atmosphere, providing data for applications in radiation thermometry. Our instrument employs a split furnace, a sample-blackbody component, two custom designed radiometers and a controlled atmospheric system. We measure across the temperature range from 973 to 1423 K and spectral range from 0.85 to 1.1 lm; this range is matched to the majority of high temperature radiation thermometers. The sample and reference approximate-blackbody are heated and maintained in thermal equilibrium, with a temperature difference of better than 1 K at 1423 K. The combined standard uncertainty of the system is lower than 0.0590 (at k=2) over the whole temperature range. Apparent emissivity of type 304 stainless steel (SS304) was studied under different oxidising procedures. Nitrogen and compressed air were input into the system to control the oxidisation process. We elucidated the relationship between the apparent emissivity variations and the surface composition changes of SS304 during oxidisation. Our study aims towards accurate and traceable apparent emissivity data, with well investigated uncertainty, for use in radiation thermometry

Artificial vision by thermography : calving prediction and defect detection in carbon fiber reinforced polymer

La vision par ordinateur est un domaine qui consiste à extraire ou identifier une ou plusieurs informations à partir d’une ou plusieurs images dans le but soit d’automatiser une tache, soit de fournir une aide à la décision. Avec l’augmentation de la capacité de calcul des ordinateurs, la vulgarisation et la diversification des moyens d’imagerie tant dans la vie quotidienne, que dans le milieu industriel,ce domaine a subi une évolution rapide lors de dernières décennies. Parmi les différentes modalités d’imagerie pour lesquels il est possible d’utiliser la vision artificielle cette thèse se concentre sur l’imagerie infrarouge. Plus particulièrement sur l’imagerie infrarouge pour les longueurs d’ondes comprises dans les bandes moyennes et longues. Cette thèse se porte sur deux applications industrielles radicalement différentes. Dans la première partie de cette thèse, nous présentons une application de la vision artificielle pour la détection du moment de vêlage en milieux industriel pour des vaches Holstein. Plus précisément l’objectif de cette recherche est de déterminer le moment de vêlage en n’utilisant que des données comportementales de l’animal. À cette fin, nous avons acquis des données en continu sur différents animaux pendant plusieurs mois. Parmi les nombreux défis présentés par cette application l’un d’entre eux concerne l’acquisition des données. En effet, les caméras que nous avons utilisées sont basées sur des capteurs bolométriques, lesquels sont sensibles à un grand nombre de variables. Ces variables peuvent être classées en quatre catégories : intrinsèque, environnemental, radiométrique et géométrique. Un autre défit important de cette recherche concerne le traitement des données. Outre le fait que les données acquises utilisent une dynamique plus élevée que les images naturelles ce qui complique le traitement des données ; l’identification de schéma récurrent dans les images et la reconnaissance automatique de ces derniers grâce à l’apprentissage automatique représente aussi un défi majeur. Nous avons proposé une solution à ce problème. Dans le reste de cette thèse nous nous sommes penchés sur la problématique de la détection de défaut dans les matériaux, en utilisant la technique de la thermographie pulsée. La thermographie pulsée est une méthode très populaire grâce à sa simplicité, la possibilité d’être utilisée avec un grand nombre de matériaux, ainsi que son faible coût. Néanmoins, cette méthode est connue pour produire des données bruitées. La cause principale de cette réputation vient des diverses sources de distorsion auquel les cameras thermiques sont sensibles. Dans cette thèse, nous avons choisi d’explorer deux axes. Le premier concerne l’amélioration des méthodes de traitement de données existantes. Dans le second axe, nous proposons plusieurs méthodes pour améliorer la détection de défauts. Chaque méthode est comparée à plusieurs méthodes constituant l’état de l’art du domaine.Abstract Computer vision is a field which consists in extracting or identifying one or more information from one or more images in order either to automate a task or to provide decision support. With the increase in the computing capacity of computers, the popularization and diversification of imaging means, both in industry, as well as in everyone’s life, this field has undergone a rapid development in recent decades. Among the different imaging modalities for which it is possible to use artificial vision, this thesis focuses on infrared imaging. More particularly on infrared imagery for wavelengths included in the medium and long bands. This thesis focuses on two radically different industrial applications. In the first part of this thesis, we present an application of artificial vision for the detection of the calving moment in industrial environments for Holstein cows. More precisely, the objective of this research is to determine the time of calving using only physiological data from the animal. To this end, we continuously acquired data on different animals over several days. Among the many challenges presented by this application, one of them concerns data acquisition. Indeed, the cameras we used are based on bolometric sensors, which are sensitive to a large number of variables. These variables can be classified into four categories: intrinsic, environmental, radiometric and geometric. Another important challenge in this research concerns the processing of data. Besides the fact that the acquired data uses a higher dynamic range than the natural images which complicates the processing of the data; Identifying recurring patterns in images and automatically recognizing them through machine learning is a major challenge. We have proposed a solution to this problem. In the rest of this thesis we have focused on the problem of defect detection in materials, using the technique of pulsed thermography. Pulse thermography is a very popular method due toits simplicity, the possibility of being used with a large number of materials, as well as its low cost. However, this method is known to produce noisy data. The main cause of this reputation comes from the various sources of distortion to which thermal cameras are sensitive. In this thesis, we have chosen to explore two axes. The first concerns the improvement of existing data processing methods. In the second axis, we propose several methods to improve fault detection. Each method is compared to several methods constituting the state of the art in the field

Design and Realisation of High Accuracy Emissivity Measurement Instruments for Radiation Thermometry

Emissivity is the quantity representing the radiative properties of materials that must be prior measured precisely to undertake accurate measurements for radiation thermometry. This work presents the development and validation of three emissivity measurement instruments to undertake studies on emissivity behaviours for materials with complex surface conditions from 200 to 1150 °C. These instruments aim to offer accurate emissivity references for use in non-contact temperature measurements and materials science

Modelling the drift of thermographic sensors UAV for efficient irrigation management

El uso civil de plataformas aéreas no tripuladas ha experimentado un notable aumento en la última década, siendo la agricultura una de las áreas que mayor interés ha despertado. La flexibilidad que ofrecen estas plataformas, permitiendo realizar vuelos sobre el cultivo en el preciso omento de interés generando estudios multi-temporales con técnicas de teledetección de muy alta resolución espacial. Estas aplicaciones están siendo posibles por la miniaturización de sensores que hace posible embarcarlos como carga de pago en estas plataformas. De este modo los sensores registran información de los cultivos en distintas regiones del espectro electromagnético, que es procesada para aplicaciones de agricultura de precisión. En función del tipo de sensor, su uso presenta un mayor o menor grado de madurez, beneficiando o limitando su uso. En el caso del uso de sensores termográficos, su uso aparece más limitado a consecuencia de la tecnología empleada si bien despierta un elevado interés tanto para su aplicación en la detección de enfermedades o evaluación de estrés hídrico en cultivos. Los sensores termográficos de uso civil se basan en una tecnología de microbolómetros no refrigerados, la cual presenta cambios continuos en la medida de temperatura. Esta inestabilidad genera una deriva en la adquisición de los valores de temperatura que debe ser corregida. Se presenta un método que permite calcular la deriva de cualquier sensor termográfico en función del tiempo.The civilian use of unmanned aerial platforms has experienced a remarkable interest in the last decade, with agriculture being one of the areas that has aroused most interest. The flexibility offered by these platforms, allowing flights over the crop at the precise moment of interest, makes it possible to carry out multi-temporal studies applying remote sensing techniques with very high spatial resolution. These applications are being made possible by the miniaturisation of sensors, which makes it possible to ship them as payloads on these platforms. In this way, sensors record crop information in different regions of the electromagnetic spectrum, which, once processed, are used in precision agriculture applications. Depending on the type of sensor, its use has a greater or lesser degree of maturity, benefiting or limiting its use. In the case of thermographic sensors, their use is more limited due to the technology used, although they are of great interest for their application in the detection of diseases or the evaluation of water stress in crops. Thermographic sensors for civilian use are based on uncooled microbolometer technology, which shows continuous changes in temperature measurement. This instability generates a drift in the acquisition of temperature values that must be corrected. A method is presented that allows the drift of any thermographic sensor to be calculated as a function of time

Cognitive and Brain-inspired Processing Using Parallel Algorithms and Heterogeneous Chip Multiprocessor Architectures

This thesis explores how some neuromorphic engineering approaches can be used to speed up computations and reduce power consumption using neuromorphic hardware systems. These hardware designs are not well-suited to conventional algorithms, so new approaches must be used to take advantage of the parallel nature of these architectures. Background regarding probabilistic graphical models is presented along with brain-inspired ways to perform inference in Bayesian networks. A spiking neuron implementation is developed on two general-purpose parallel neuromorphic hardware devices, the SpiNNaker and the Parallella. Scalability results are shown along with speed improvements as compared to using mainstream processors on a desktop computer. General vector-matrix multiplication computations at various levels of precision are also explored using IBM's TrueNorth Neurosynaptic System. The TrueNorth contains highly-configurable hardware neurons and axons connected via crossbar arrays and consumes very little power but is less flexible than a more general-purpose neuromorphic system such as the SpiNNaker. Nevertheless, techniques described here enable useful computations to be performed utilizing such crossbar arrays with spiking neurons including computing word similarities using trained word vector embeddings. Another technique describes how to perform computations using only one column of the crossbar array at a time despite the fact that incoming spikes normally affect all columns of the array. A way to perform cognitive audio-visual beamforming is presented. Using two systems, each containing a spherical microphone array, sounds are localized using spherical harmonic beamforming. Combining the microphone arrays with 360 degree cameras provides an opportunity to overlay the sound localization with the visual data and create a combined audio-visual salience map. Cognitive computations can be performed on the audio signals to localize specific sounds while ignoring others based on their spectral characteristics. Finally, an ARM Cortex M0 processor design is shown that will be used to bootstrap and coordinate other processing units on a chip developed in the lab for the DARPA Unconventional Processing of Signals for Intelligent Data Exploitation (UPSIDE) program. This design includes a bootloader which provides full programmability each time the chip is booted, and the processor interfaces with other hardware modules to access the Networks-on-Chip and main memory