Assessment of registration methods for thermal infrared and visible images for diabetic foot monitoring

This work presents a revision of four different registration methods for thermal infrared and visible images captured by a camera-based prototype for the remote monitoring of diabetic foot. This prototype uses low cost and off-the-shelf available sensors in thermal infrared and visible spectra. Four different methods (Geometric Optical Translation, Homography, Iterative Closest Point, and Affine transform with Gradient Descent) have been implemented and analyzed for the registration of images obtained from both sensors. All four algorithms´ performances were evaluated using the Simultaneous Truth and Performance Level Estimation (STAPLE) together with several overlap benchmarks as the Dice coefficient and the Jaccard index. The performance of the four methods has been analyzed with the subject at a fixed focal plane and also in the vicinity of this plane. The four registration algorithms provide suitable results both at the focal plane as well as outside of it within 50 mm margin. The obtained Dice coefficients are greater than 0.950 in all scenarios, well within the margins required for the application at hand. A discussion of the obtained results under different distances is presented along with an evaluation of its robustness under changing conditions.This research was funded by the IACTEC Technological Training program, grant number TF INNOVA 2016–2021

NIR Imagery-based Grass Fire Detection and Metrics Measurement using Small UAS

This thesis focuses on the generation of a new grass fire aerial image dataset and development of novel methods for near-infrared (NIR) imagery-based fire front identification and fire depth estimation using small unmanned aircraft systems (sUAS). The procedure for collection and creation of the Grass Fire Front and near-Infrared (NIR) and Thermal Imagery (GRAFFITI) dataset is introduced first including two levels of data: synced raw thermal and red, green and near-infrared (RGNIR) image pairs and processed image pairs of the same overlapping field-of-view. A novel NIR imagery-based fire detection and fire front identification algorithm is then proposed and validated against manually labeled ground truth, using the GRAFFITI dataset. A comparative study is further performed on the problem of grass fire front location and flame depth estimation using thermal and NIR imagery. Finally, recommendations are made to future researchers who are interested in wildland fire sensing using thermal or NIR imagery

Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electrooptical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The arti

Object Detection and Classification in the Visible and Infrared Spectrums

The over-arching theme of this dissertation is the development of automated detection and/or classification systems for challenging infrared scenarios. The six works presented herein can be categorized into four problem scenarios. In the first scenario, long-distance detection and classification of vehicles in thermal imagery, a custom convolutional network architecture is proposed for small thermal target detection. For the second scenario, thermal face landmark detection and thermal cross-spectral face verification, a publicly-available visible and thermal face dataset is introduced, along with benchmark results for several landmark detection and face verification algorithms. Furthermore, a novel visible-to-thermal transfer learning algorithm for face landmark detection is presented. The third scenario addresses near-infrared cross-spectral periocular recognition with a coupled conditional generative adversarial network guided by auxiliary synthetic loss functions. Finally, a deep sparse feature selection and fusion is proposed to detect the presence of textured contact lenses prior to near-infrared iris recognition

Simulation of a new respiratory phase sorting method for 4D-imaging using optical surface information towards precision radiotherapy

Background: Respiratory signal detection is critical for 4-dimensional (4D) imaging. This study proposes and evaluates a novel phase sorting method using optical surface imaging (OSI), aiming to improve the precision of radiotherapy. Method: Based on 4D Extended Cardiac-Torso (XCAT) digital phantom, OSI in point cloud format was generated from the body segmentation, and image projections were simulated using the geometries of Varian 4D kV cone-beam-CT (CBCT). Respiratory signals were extracted respectively from the segmented diaphragm image (reference method) and OSI respectively, where Gaussian Mixture Model and Principal Component Analysis (PCA) were used for image registration and dimension reduction respectively. Breathing frequencies were compared using Fast-Fourier-Transform. Consistency of 4DCBCT images reconstructed using Maximum Likelihood Expectation Maximization algorithm was also evaluated quantitatively, where high consistency can be suggested by lower Root-Mean-Square-Error (RMSE), Structural-Similarity-Index (SSIM) value closer to 1, and larger Peak-Signal-To-Noise-Ratio (PSNR) respectively. Results: High consistency of breathing frequencies was observed between the diaphragm-based (0.232 Hz) and OSI-based (0.251 Hz) signals, with a slight discrepancy of 0.019Hz. Using end of expiration (EOE) and end of inspiration (EOI) phases as examples, the mean±1SD values of the 80 transverse, 100 coronal and 120 sagittal planes were 0.967, 0,972, 0.974 (SSIM); 1.657 ± 0.368, 1.464 ± 0.104, 1.479 ± 0.297 (RMSE); and 40.501 ± 1.737, 41.532 ± 1.464, 41.553 ± 1.910 (PSNR) for the EOE; and 0.969, 0.973, 0.973 (SSIM); 1.686 ± 0.278, 1.422 ± 0.089, 1.489 ± 0.238 (RMSE); and 40.535 ± 1.539, 41.605 ± 0.534, 41.401 ± 1.496 (PSNR) for EOI respectively. Conclusions: This work proposed and evaluated a novel respiratory phase sorting approach for 4D imaging using optical surface signals, which can potentially be applied to precision radiotherapy. Its potential advantages were non-ionizing, non-invasive, non-contact, and more compatible with various anatomic regions and treatment/imaging systems

Benchmarking of mobile phone cameras

fi=vertaisarvioitu|en=peerReviewed

Multispectral Remote Sensing of the Earth and Environment Using KHawk Unmanned Aircraft Systems

This thesis focuses on the development and testing of the KHawk multispectral remote sensing system for environmental and agricultural applications. KHawk Unmanned Aircraft System (UAS), a small and low-cost remote sensing platform, is used as the test bed for aerial video acquisition. An efficient image geotagging and photogrammetric procedure for aerial map generation is described, followed by a comprehensive error analysis on the generated maps. The developed procedure is also used for generation of multispectral aerial maps including red, near infrared (NIR) and colored infrared (CIR) maps. A robust Normalized Difference Vegetation index (NDVI) calibration procedure is proposed and validated by ground tests and KHawk flight test. Finally, the generated aerial maps and their corresponding Digital Elevation Models (DEMs) are used for typical application scenarios including prescribed fire monitoring, initial fire line estimation, and tree health monitoring

Requirement analysis and sensor specifications – First version

In this first version of the deliverable, we make the following contributions: to design the WEKIT capturing platform and the associated experience capturing API, we use a methodology for system engineering that is relevant for different domains such as: aviation, space, and medical and different professions such as: technicians, astronauts, and medical staff. Furthermore, in the methodology, we explore the system engineering process and how it can be used in the project to support the different work packages and more importantly the different deliverables that will follow the current. Next, we provide a mapping of high level functions or tasks (associated with experience transfer from expert to trainee) to low level functions such as: gaze, voice, video, body posture, hand gestures, bio-signals, fatigue levels, and location of the user in the environment. In addition, we link the low level functions to their associated sensors. Moreover, we provide a brief overview of the state-of-the-art sensors in terms of their technical specifications, possible limitations, standards, and platforms. We outline a set of recommendations pertaining to the sensors that are most relevant for the WEKIT project taking into consideration the environmental, technical and human factors described in other deliverables. We recommend Microsoft Hololens (for Augmented reality glasses), MyndBand and Neurosky chipset (for EEG), Microsoft Kinect and Lumo Lift (for body posture tracking), and Leapmotion, Intel RealSense and Myo armband (for hand gesture tracking). For eye tracking, an existing eye-tracking system can be customised to complement the augmented reality glasses, and built-in microphone of the augmented reality glasses can capture the expert’s voice. We propose a modular approach for the design of the WEKIT experience capturing system, and recommend that the capturing system should have sufficient storage or transmission capabilities. Finally, we highlight common issues associated with the use of different sensors. We consider that the set of recommendations can be useful for the design and integration of the WEKIT capturing platform and the WEKIT experience capturing API to expedite the time required to select the combination of sensors which will be used in the first prototype.WEKI