A mask-based approach for the geometric calibration of thermal-infrared cameras

Accurate and efficient thermal-infrared (IR) camera calibration is important for advancing computer vision research within the thermal modality. This paper presents an approach for geometrically calibrating individual and multiple cameras in both the thermal and visible modalities. The proposed technique can be used to correct for lens distortion and to simultaneously reference both visible and thermal-IR cameras to a single coordinate frame. The most popular existing approach for the geometric calibration of thermal cameras uses a printed chessboard heated by a flood lamp and is comparatively inaccurate and difficult to execute. Additionally, software toolkits provided for calibration either are unsuitable for this task or require substantial manual intervention. A new geometric mask with high thermal contrast and not requiring a flood lamp is presented as an alternative calibration pattern. Calibration points on the pattern are then accurately located using a clustering-based algorithm which utilizes the maximally stable extremal region detector. This algorithm is integrated into an automatic end-to-end system for calibrating single or multiple cameras. The evaluation shows that using the proposed mask achieves a mean reprojection error up to 78% lower than that using a heated chessboard. The effectiveness of the approach is further demonstrated by using it to calibrate two multiple-camera multiple-modality setups. Source code and binaries for the developed software are provided on the project Web site

Tracing The Origins And Propagation Of African Easterly Waves And Mesoscale Convective Systems Using Pattern Recognition And Data Fusion

This thesis is focus on developing pattern recognition techniques to trace the origins and propagation of the Pre-Tropical Storm Debby (2006) African easterly waves (AEWs) and mesoscale convective systems (MCSs) using satellite imagery. The results are used to verify a numerical weather prediction (NWP) model. The pre-Debby MCSs’ movement and formation needs to be precisely and objectively tracked. These MCSs could be generated over mountains in North Africa, such as Ethiopian Highlands (EH), Darfur Mountains (DF), and Asir Mountains (AS), going through complicated splitting and merging processes. Thus, an objectively analyzed MCS movement is essential not only to help provide data to verify numerical modeling results, but also to help understand the formation and propagation of the African easterly waves and MCSs. The technique used could be applied to other AEWs and MCSs leading to tropical cyclogenesis. This, in turn, will improve the NWP over the data sparse areas, such as over eastern and central North Africa. The accuracy of numerical simulations of pre-tropical cyclone (TC) AEWs and MCSs is improved by fusing data with different data fusion techniques. In order to provide enhanced information to help predict the weather, this thesis investigates various techniques of data fusion. In many cases, fusing data provides more accurate and complete initial data for the NWP models which will then reduce the errors in weather prediction

Realistic texture in simulated thermal infrared imagery

Creating a visually-realistic yet radiometrically-accurate simulation of thermal infrared (TIR) imagery is a challenge that has plagued members of industry and academia alike. The goal of imagery simulation is to provide a practical alternative to the often staggering effort required to collect actual data. Previous attempts at simulating TIR imagery have suffered from a lack of texture—the simulated scenes generally failed to reproduce the natural variability seen in actual TIR images. Realistic synthetic TIR imagery requires modeling sources of variability including surface effects such as solar insolation and convective heat exchange as well as sub-surface effects such as density and water content. This research effort utilized the Digital Imaging and Remote Sensing Image Generation (DIRSIG) model, developed at the Rochester Institute of Technology, to investigate how these additional sources of variability could be modeled to correctly and accurately provide simulated TIR imagery. Actual thermal data were collected, analyzed, and exploited to determine the underlying thermodynamic phenomena and ascertain how these phenomena are best modeled. The underlying task was to determine how to apply texture in the thermal region to attain radiometrically-correct, visually-appealing simulated imagery. Three natural desert scenes were used to test the methodologies that were developed for estimating per-pixel thermal parameters which could then be used for TIR image simulation by DIRSIG. Additional metrics were devised and applied to the synthetic images to further quantify the success of this research. The resulting imagery demonstrated that these new methodologies for modeling TIR phenomena and the utilization of an improved DIRSIG tool improved the root mean-squared error (RMSE) of our synthetic TIR imagery by up to 88%

Classification of non-heat generating outdoor objects in thermal scenes for autonomous robots

We have designed and implemented a physics-based adaptive Bayesian pattern classification model that uses a passive thermal infrared imaging system to automatically characterize non-heat generating objects in unstructured outdoor environments for mobile robots. In the context of this research, non-heat generating objects are defined as objects that are not a source for their own emission of thermal energy, and so exclude people, animals, vehicles, etc. The resulting classification model complements an autonomous bot\u27s situational awareness by providing the ability to classify smaller structures commonly found in the immediate operational environment. Since GPS depends on the availability of satellites and onboard terrain maps which are often unable to include enough detail for smaller structures found in an operational environment, bots will require the ability to make decisions such as go through the hedges or go around the brick wall. A thermal infrared imaging modality mounted on a small mobile bot is a favorable choice for receiving enough detailed information to automatically interpret objects at close ranges while unobtrusively traveling alongside pedestrians. The classification of indoor objects and heat generating objects in thermal scenes is a solved problem. A missing and essential piece in the literature has been research involving the automatic characterization of non-heat generating objects in outdoor environments using a thermal infrared imaging modality for mobile bots. Seeking to classify non-heat generating objects in outdoor environments using a thermal infrared imaging system is a complex problem due to the variation of radiance emitted from the objects as a result of the diurnal cycle of solar energy. The model that we present will allow bots to see beyond vision to autonomously assess the physical nature of the surrounding structures for making decisions without the need for an interpretation by humans.;Our approach is an application of Bayesian statistical pattern classification where learning involves labeled classes of data (supervised classification), assumes no formal structure regarding the density of the data in the classes (nonparametric density estimation), and makes direct use of prior knowledge regarding an object class\u27s existence in a bot\u27s immediate area of operation when making decisions regarding class assignments for unknown objects. We have used a mobile bot to systematically capture thermal infrared imagery for two categories of non-heat generating objects (extended and compact) in several different geographic locations. The extended objects consist of objects that extend beyond the thermal camera\u27s field of view, such as brick walls, hedges, picket fences, and wood walls. The compact objects consist of objects that are within the thermal camera\u27s field of view, such as steel poles and trees. We used these large representative data sets to explore the behavior of thermal-physical features generated from the signals emitted by the classes of objects and design our Adaptive Bayesian Classification Model. We demonstrate that our novel classification model not only displays exceptional performance in characterizing non-heat generating outdoor objects in thermal scenes but it also outperforms the traditional KNN and Parzen classifiers

Incremental low rank noise reduction for robust infrared tracking of body temperature during medical imaging

Thermal imagery for monitoring of body temperature provides a powerful tool to decrease health risks (e.g., burning) for patients during medical imaging (e.g., magnetic resonance imaging). The presented approach discusses an experiment to simulate radiology conditions with infrared imaging along with an automatic thermal monitoring/tracking system. The thermal tracking system uses an incremental low-rank noise reduction applying incremental singular value decomposition (SVD) and applies color based clustering for initialization of the region of interest (ROI) boundary. Then a particle filter tracks the ROI(s) from the entire thermal stream (video sequence). The thermal database contains 15 subjects in two positions (i.e., sitting, and lying) in front of thermal camera. This dataset is created to verify the robustness of our method with respect to motion-artifacts and in presence of additive noise (2–20%—salt and pepper noise). The proposed approach was tested for the infrared images in the dataset and was able to successfully measure and track the ROI continuously (100% detecting and tracking the temperature of participants), and provided considerable robustness against noise (unchanged accuracy even in 20% additive noise), which shows promising performanc

Adaptive detection and tracking using multimodal information

This thesis describes work on fusing data from multiple sources of information, and focuses on two main areas: adaptive detection and adaptive object tracking in automated vision scenarios. The work on adaptive object detection explores a new paradigm in dynamic parameter selection, by selecting thresholds for object detection to maximise agreement between pairs of sources. Object tracking, a complementary technique to object detection, is also explored in a multi-source context and an efficient framework for robust tracking, termed the Spatiogram Bank tracker, is proposed as a means to overcome the difficulties of traditional histogram tracking. As well as performing theoretical analysis of the proposed methods, specific example applications are given for both the detection and the tracking aspects, using thermal infrared and visible spectrum video data, as well as other multi-modal information sources

Object detection, recognition and re-identification in video footage

There has been a significant number of security concerns in recent times; as a result, security cameras have been installed to monitor activities and to prevent crimes in most public places. These analysis are done either through video analytic or forensic analysis operations on human observations. To this end, within the research context of this thesis, a proactive machine vision based military recognition system has been developed to help monitor activities in the military environment. The proposed object detection, recognition and re-identification systems have been presented in this thesis. A novel technique for military personnel recognition is presented in this thesis. Initially the detected camouflaged personnel are segmented using a grabcut segmentation algorithm. Since in general a camouflaged personnel's uniform appears to be similar both at the top and the bottom of the body, an image patch is initially extracted from the segmented foreground image and used as the region of interest. Subsequently the colour and texture features are extracted from each patch and used for classification. A second approach for personnel recognition is proposed through the recognition of the badge on the cap of a military person. A feature matching metric based on the extracted Speed Up Robust Features (SURF) from the badge on a personnel's cap enabled the recognition of the personnel's arm of service. A state-of-the-art technique for recognising vehicle types irrespective of their view angle is also presented in this thesis. Vehicles are initially detected and segmented using a Gaussian Mixture Model (GMM) based foreground/background segmentation algorithm. A Canny Edge Detection (CED) stage, followed by morphological operations are used as pre-processing stage to help enhance foreground vehicular object detection and segmentation. Subsequently, Region, Histogram Oriented Gradient (HOG) and Local Binary Pattern (LBP) features are extracted from the refined foreground vehicle object and used as features for vehicle type recognition. Two different datasets with variant views of front/rear and angle are used and combined for testing the proposed technique. For night-time video analytics and forensics, the thesis presents a novel approach to pedestrian detection and vehicle type recognition. A novel feature acquisition technique named, CENTROG, is proposed for pedestrian detection and vehicle type recognition in this thesis. Thermal images containing pedestrians and vehicular objects are used to analyse the performance of the proposed algorithms. The video is initially segmented using a GMM based foreground object segmentation algorithm. A CED based pre-processing step is used to enhance segmentation accuracy prior using Census Transforms for initial feature extraction. HOG features are then extracted from the Census transformed images and used for detection and recognition respectively of human and vehicular objects in thermal images. Finally, a novel technique for people re-identification is proposed in this thesis based on using low-level colour features and mid-level attributes. The low-level colour histogram bin values were normalised to 0 and 1. A publicly available dataset (VIPeR) and a self constructed dataset have been used in the experiments conducted with 7 clothing attributes and low-level colour histogram features. These 7 attributes are detected using features extracted from 5 different regions of a detected human object using an SVM classifier. The low-level colour features were extracted from the regions of a detected human object. These 5 regions are obtained by human object segmentation and subsequent body part sub-division. People are re-identified by computing the Euclidean distance between a probe and the gallery image sets. The experiments conducted using SVM classifier and Euclidean distance has proven that the proposed techniques attained all of the aforementioned goals. The colour and texture features proposed for camouflage military personnel recognition surpasses the state-of-the-art methods. Similarly, experiments prove that combining features performed best when recognising vehicles in different views subsequent to initial training based on multi-views. In the same vein, the proposed CENTROG technique performed better than the state-of-the-art CENTRIST technique for both pedestrian detection and vehicle type recognition at night-time using thermal images. Finally, we show that the proposed 7 mid-level attributes and the low-level features results in improved performance accuracy for people re-identification

Crop Disease Detection Using Remote Sensing Image Analysis

Pest and crop disease threats are often estimated by complex changes in crops and the applied agricultural practices that result mainly from the increasing food demand and climate change at global level. In an attempt to explore high-end and sustainable solutions for both pest and crop disease management, remote sensing technologies have been employed, taking advantages of possible changes deriving from relative alterations in the metabolic activity of infected crops which in turn are highly associated to crop spectral reflectance properties. Recent developments applied to high resolution data acquired with remote sensing tools, offer an additional tool which is the opportunity of mapping the infected field areas in the form of patchy land areas or those areas that are susceptible to diseases. This makes easier the discrimination between healthy and diseased crops, providing an additional tool to crop monitoring. The current book brings together recent research work comprising of innovative applications that involve novel remote sensing approaches and their applications oriented to crop disease detection. The book provides an in-depth view of the developments in remote sensing and explores its potential to assess health status in crops

LANDSAT-D investigations in snow hydrology

Work undertaken during the contract and its results are described. Many of the results from this investigation are available in journal or conference proceedings literature - published, accepted for publication, or submitted for publication. For these the reference and the abstract are given. Those results that have not yet been submitted separately for publication are described in detail. Accomplishments during the contract period are summarized as follows: (1) analysis of the snow reflectance characteristics of the LANDSAT Thematic Mapper, including spectral suitability, dynamic range, and spectral resolution; (2) development of a variety of atmospheric models for use with LANDSAT Thematic Mapper data. These include a simple but fast two-stream approximation for inhomogeneous atmospheres over irregular surfaces, and a doubling model for calculation of the angular distribution of spectral radiance at any level in an plane-parallel atmosphere; (3) incorporation of digital elevation data into the atmospheric models and into the analysis of the satellite data; and (4) textural analysis of the spatial distribution of snow cover

Signal processing algorithms for enhanced image fusion performance and assessment

The dissertation presents several signal processing algorithms for image fusion in noisy multimodal conditions. It introduces a novel image fusion method which performs well for image sets heavily corrupted by noise. As opposed to current image fusion schemes, the method has no requirements for a priori knowledge of the noise component. The image is decomposed with Chebyshev polynomials (CP) being used as basis functions to perform fusion at feature level. The properties of CP, namely fast convergence and smooth approximation, renders it ideal for heuristic and indiscriminate denoising fusion tasks. Quantitative evaluation using objective fusion assessment methods show favourable performance of the proposed scheme compared to previous efforts on image fusion, notably in heavily corrupted images. The approach is further improved by incorporating the advantages of CP with a state-of-the-art fusion technique named independent component analysis (ICA), for joint-fusion processing based on region saliency. Whilst CP fusion is robust under severe noise conditions, it is prone to eliminating high frequency information of the images involved, thereby limiting image sharpness. Fusion using ICA, on the other hand, performs well in transferring edges and other salient features of the input images into the composite output. The combination of both methods, coupled with several mathematical morphological operations in an algorithm fusion framework, is considered a viable solution. Again, according to the quantitative metrics the results of our proposed approach are very encouraging as far as joint fusion and denoising are concerned. Another focus of this dissertation is on a novel metric for image fusion evaluation that is based on texture. The conservation of background textural details is considered important in many fusion applications as they help define the image depth and structure, which may prove crucial in many surveillance and remote sensing applications. Our work aims to evaluate the performance of image fusion algorithms based on their ability to retain textural details from the fusion process. This is done by utilising the gray-level co-occurrence matrix (GLCM) model to extract second-order statistical features for the derivation of an image textural measure, which is then used to replace the edge-based calculations in an objective-based fusion metric. Performance evaluation on established fusion methods verifies that the proposed metric is viable, especially for multimodal scenarios