Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns

We introduce Deep Thermal Imaging, a new approach for close-range automatic recognition of materials to enhance the understanding of people and ubiquitous technologies of their proximal environment. Our approach uses a low-cost mobile thermal camera integrated into a smartphone to capture thermal textures. A deep neural network classifies these textures into material types. This approach works effectively without the need for ambient light sources or direct contact with materials. Furthermore, the use of a deep learning network removes the need to handcraft the set of features for different materials. We evaluated the performance of the system by training it to recognise 32 material types in both indoor and outdoor environments. Our approach produced recognition accuracies above 98% in 14,860 images of 15 indoor materials and above 89% in 26,584 images of 17 outdoor materials. We conclude by discussing its potentials for real-time use in HCI applications and future directions.Comment: Proceedings of the 2018 CHI Conference on Human Factors in Computing System

A Model that Predicts the Material Recognition Performance of Thermal Tactile Sensing

Tactile sensing can enable a robot to infer properties of its surroundings, such as the material of an object. Heat transfer based sensing can be used for material recognition due to differences in the thermal properties of materials. While data-driven methods have shown promise for this recognition problem, many factors can influence performance, including sensor noise, the initial temperatures of the sensor and the object, the thermal effusivities of the materials, and the duration of contact. We present a physics-based mathematical model that predicts material recognition performance given these factors. Our model uses semi-infinite solids and a statistical method to calculate an F1 score for the binary material recognition. We evaluated our method using simulated contact with 69 materials and data collected by a real robot with 12 materials. Our model predicted the material recognition performance of support vector machine (SVM) with 96% accuracy for the simulated data, with 92% accuracy for real-world data with constant initial sensor temperatures, and with 91% accuracy for real-world data with varied initial sensor temperatures. Using our model, we also provide insight into the roles of various factors on recognition performance, such as the temperature difference between the sensor and the object. Overall, our results suggest that our model could be used to help design better thermal sensors for robots and enable robots to use them more effectively.Comment: This article is currently under review for possible publicatio

Observational Characterization of the Downward Atmospheric Longwave Radiation at the Surface in the City of São Paulo

This work describes the seasonal and diurnal variations of downward longwave atmospheric irradiance (LW) at the surface in São Paulo, Brazil, using 5-min-averaged values of LW, air temperature, relative humidity, and solar radiation observed continuously and simultaneously from 1997 to 2006 on a micrometeorological platform, located at the top of a 4-story building. An objective procedure, including 2-step filtering and dome emission effect correction, was used to evaluate the quality of the 9-yr-long LW dataset. The comparison between LW values observed and yielded by the Surface Radiation Budget project shows spatial and temporal agreement, indicating that monthly and annual average values of LW observed in one point of São Paulo can be used as representative of the entire metropolitan region of São Paulo. The maximum monthly averaged value of the LW is observed during summer (389 ± 14 W m-2; January), and the minimum is observed during winter (332 ± 12 W m-2; July). The effective emissivity follows the LW and shows a maximum in summer (0.907 ± 0.032; January) and a minimum in winter (0.818 ± 0.029; June). The mean cloud effect, identified objectively by comparing the monthly averaged values of the LW during clear-sky days and all-sky conditions, intensified the monthly average LW by about 32.0 ± 3.5 W m-2 and the atmospheric effective emissivity by about 0.088 ± 0.024. In August, the driest month of the year in São Paulo, the diurnal evolution of the LW shows a minimum (325 ± 11 W m-2) at 0900 LT and a maximum (345 ± 12 W m-2) at 1800 LT, which lags behind (by 4 h) the maximum diurnal variation of the screen temperature. The diurnal evolution of effective emissivity shows a minimum (0.781 ± 0.027) during daytime and a maximum (0.842 ± 0.030) during nighttime. The diurnal evolution of all-sky condition and clear-sky day differences in the effective emissivity remain relatively constant (7% ± 1%), indicating that clouds do not change the emissivity diurnal pattern. The relationship between effective emissivity and screen air temperature and between effective emissivity and water vapor is complex. During the night, when the planetary boundary layer is shallower, the effective emissivity can be estimated by screen parameters. During the day, the relationship between effective emissivity and screen parameters varies from place to place and depends on the planetary boundary layer process. Because the empirical expressions do not contain enough information about the diurnal variation of the vertical stratification of air temperature and moisture in São Paulo, they are likely to fail in reproducing the diurnal variation of the surface emissivity. The most accurate way to estimate the LW for clear-sky conditions in São Paulo is to use an expression derived from a purely empirical approach

The Effect of a Diverse Dataset for Transfer Learning in Thermal Person Detection

Thermal cameras are popular in detection for their precision in surveillance in the dark and for privacy preservation. In the era of data driven problem solving approaches, manually finding and annotating a large amount of data is inefficient in terms of cost and effort. With the introduction of transfer learning, rather than having large datasets, a dataset covering all characteristics and aspects of the target place is more important. In this work, we studied a large thermal dataset recorded for 20 weeks and identified nine phenomena in it. Moreover, we investigated the impact of each phenomenon for model adaptation in transfer learning. Each phenomenon was investigated separately and in combination. the performance was analyzed by computing the F1 score, precision, recall, true negative rate, and false negative rate. Furthermore, to underline our investigation, the trained model with our dataset was further tested on publicly available datasets, and encouraging results were obtained. Finally, our dataset was also made publicly available

Selected bibliography of remote sensing

Bibliography of remote sensing techniques for analysis and assimilation of geographic dat

Climatology of Urban-regional Systems

Urbanized areas have come to be significant if not dominant components of many regional land surfaces. They represent perhaps the most dramatic recent change man has made in his environment - a change that may well burgeon in the foreseeable future as greater percentages of world populations crowd into metropolitan areas. The climate of urban-regional systems is involved because temperature, air, and pollutants added to the air are significant aspects of this change. During the past two years, substantial progress has been made in the application of remote sensing techniques to the study of urban climatology by programs jointly sponsored by NASA and the United States Geological Survey. The initial effort has endeavored with considerable success to map terrestrial radiation emission or the general thermal state of the land surface with the aid of imaging radiometers (mechanical-optical scanners)