Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

Effective fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g. daytime and nighttime). In this paper, we present a novel box-level segmentation supervised learning framework for accurate and real-time multispectral pedestrian detection by incorporating features extracted in visible and infrared channels. Specifically, our method takes pairs of aligned visible and infrared images with easily obtained bounding box annotations as input and estimates accurate prediction maps to highlight the existence of pedestrians. It offers two major advantages over the existing anchor box based multispectral detection methods. Firstly, it overcomes the hyperparameter setting problem occurred during the training phase of anchor box based detectors and can obtain more accurate detection results, especially for small and occluded pedestrian instances. Secondly, it is capable of generating accurate detection results using small-size input images, leading to improvement of computational efficiency for real-time autonomous driving applications. Experimental results on KAIST multispectral dataset show that our proposed method outperforms state-of-the-art approaches in terms of both accuracy and speed

Infrared face recognition: a comprehensive review of methodologies and databases

Automatic face recognition is an area with immense practical potential which includes a wide range of commercial and law enforcement applications. Hence it is unsurprising that it continues to be one of the most active research areas of computer vision. Even after over three decades of intense research, the state-of-the-art in face recognition continues to improve, benefitting from advances in a range of different research fields such as image processing, pattern recognition, computer graphics, and physiology. Systems based on visible spectrum images, the most researched face recognition modality, have reached a significant level of maturity with some practical success. However, they continue to face challenges in the presence of illumination, pose and expression changes, as well as facial disguises, all of which can significantly decrease recognition accuracy. Amongst various approaches which have been proposed in an attempt to overcome these limitations, the use of infrared (IR) imaging has emerged as a particularly promising research direction. This paper presents a comprehensive and timely review of the literature on this subject. Our key contributions are: (i) a summary of the inherent properties of infrared imaging which makes this modality promising in the context of face recognition, (ii) a systematic review of the most influential approaches, with a focus on emerging common trends as well as key differences between alternative methodologies, (iii) a description of the main databases of infrared facial images available to the researcher, and lastly (iv) a discussion of the most promising avenues for future research.Comment: Pattern Recognition, 2014. arXiv admin note: substantial text overlap with arXiv:1306.160

Time frequency analysis in terahertz pulsed imaging

Recent advances in laser and electro-optical technologies have made the previously under-utilized terahertz frequency band of the electromagnetic spectrum accessible for practical imaging. Applications are emerging, notably in the biomedical domain. In this chapter the technique of terahertz pulsed imaging is introduced in some detail. The need for special computer vision methods, which arises from the use of pulses of radiation and the acquisition of a time series at each pixel, is described. The nature of the data is a challenge since we are interested not only in the frequency composition of the pulses, but also how these differ for different parts of the pulse. Conventional and short-time Fourier transforms and wavelets were used in preliminary experiments on the analysis of terahertz pulsed imaging data. Measurements of refractive index and absorption coefficient were compared, wavelet compression assessed and image classification by multidimensional clustering techniques demonstrated. It is shown that the timefrequency methods perform as well as conventional analysis for determining material properties. Wavelet compression gave results that were robust through compressions that used only 20% of the wavelet coefficients. It is concluded that the time-frequency methods hold great promise for optimizing the extraction of the spectroscopic information contained in each terahertz pulse, for the analysis of more complex signals comprising multiple pulses or from recently introduced acquisition techniques

Volcanic Hot-Spot Detection Using SENTINEL-2: A Comparison with MODIS−MIROVA Thermal Data Series

In the satellite thermal remote sensing, the new generation of sensors with high-spatial resolution SWIR data open the door to an improved constraining of thermal phenomena related to volcanic processes, with strong implications for monitoring applications. In this paper, we describe a new hot-spot detection algorithm developed for SENTINEL-2/MSI data that combines spectral indices on the SWIR bands 8a-11-12 (with a 20-meter resolution) with a spatial and statistical analysis on clusters of alerted pixels. The algorithm is able to detect hot-spot-contaminated pixels (S2Pix) in a wide range of environments and for several types of volcanic activities, showing high accuracy performances of about 1% and 94% in averaged omission and commission rates, respectively, underlining a strong reliability on a global scale. The S2-derived thermal trends, retrieved at eight key-case volcanoes, are then compared with the Volcanic Radiative Power (VRP) derived from MODIS (Moderate Resolution Imaging Spectroradiometer) and processed by the MIROVA (Middle InfraRed Observation of Volcanic Activity) system during an almost four-year-long period, January 2016 to October 2019. The presented data indicate an overall excellent correlation between the two thermal signals, enhancing the higher sensitivity of SENTINEL-2 to detect subtle, low-temperature thermal signals. Moreover, for each case we explore the specific relationship between S2Pix and VRP showing how different volcanic processes (i.e., lava flows, domes, lakes and open-vent activity) produce a distinct pattern in terms of size and intensity of the thermal anomaly. These promising results indicate how the algorithm here presented could be applicable for volcanic monitoring purposes and integrated into operational systems. Moreover, the combination of high-resolution (S2/MSI) and moderate-resolution (MODIS) thermal timeseries constitutes a breakthrough for future multi-sensor hot-spot detection systems, with increased monitoring capabilities that are useful for communities which interact with active volcanoes

Micro-Doppler Based Human-Robot Classification Using Ensemble and Deep Learning Approaches

Radar sensors can be used for analyzing the induced frequency shifts due to micro-motions in both range and velocity dimensions identified as micro-Doppler ($\boldsymbol{\mu}$-D) and micro-Range ($\boldsymbol{\mu}$-R), respectively. Different moving targets will have unique $\boldsymbol{\mu}$-D and $\boldsymbol{\mu}$-R signatures that can be used for target classification. Such classification can be used in numerous fields, such as gait recognition, safety and surveillance. In this paper, a 25 GHz FMCW Single-Input Single-Output (SISO) radar is used in industrial safety for real-time human-robot identification. Due to the real-time constraint, joint Range-Doppler (R-D) maps are directly analyzed for our classification problem. Furthermore, a comparison between the conventional classical learning approaches with handcrafted extracted features, ensemble classifiers and deep learning approaches is presented. For ensemble classifiers, restructured range and velocity profiles are passed directly to ensemble trees, such as gradient boosting and random forest without feature extraction. Finally, a Deep Convolutional Neural Network (DCNN) is used and raw R-D images are directly fed into the constructed network. DCNN shows a superior performance of 99\% accuracy in identifying humans from robots on a single R-D map.Comment: 6 pages, accepted in IEEE Radar Conference 201