Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection

Multispectral pedestrian detection has received extensive attention in recent years as a promising solution to facilitate robust human target detection for around-the-clock applications (e.g. security surveillance and autonomous driving). In this paper, we demonstrate illumination information encoded in multispectral images can be utilized to significantly boost performance of pedestrian detection. A novel illumination-aware weighting mechanism is present to accurately depict illumination condition of a scene. Such illumination information is incorporated into two-stream deep convolutional neural networks to learn multispectral human-related features under different illumination conditions (daytime and nighttime). Moreover, we utilized illumination information together with multispectral data to generate more accurate semantic segmentation which are used to boost pedestrian detection accuracy. Putting all of the pieces together, we present a powerful framework for multispectral pedestrian detection based on multi-task learning of illumination-aware pedestrian detection and semantic segmentation. Our proposed method is trained end-to-end using a well-designed multi-task loss function and outperforms state-of-the-art approaches on KAIST multispectral pedestrian dataset

Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

Effective fusion of complementary information captured by multi-modal sensors (visible and infrared cameras) enables robust pedestrian detection under various surveillance situations (e.g. daytime and nighttime). In this paper, we present a novel box-level segmentation supervised learning framework for accurate and real-time multispectral pedestrian detection by incorporating features extracted in visible and infrared channels. Specifically, our method takes pairs of aligned visible and infrared images with easily obtained bounding box annotations as input and estimates accurate prediction maps to highlight the existence of pedestrians. It offers two major advantages over the existing anchor box based multispectral detection methods. Firstly, it overcomes the hyperparameter setting problem occurred during the training phase of anchor box based detectors and can obtain more accurate detection results, especially for small and occluded pedestrian instances. Secondly, it is capable of generating accurate detection results using small-size input images, leading to improvement of computational efficiency for real-time autonomous driving applications. Experimental results on KAIST multispectral dataset show that our proposed method outperforms state-of-the-art approaches in terms of both accuracy and speed

Image-to-Image Training for Spatially Seamless Air Temperature Estimation with Satellite Images and Station Data

Air temperature at approximately 2 m above the ground (T-a) is one of the most important environmental and biophysical parameters to study various earth surface processes. T-a measured from meteorological stations is inadequate to study its spatio-temporal patterns since the stations are unevenly and sparsely distributed. Satellite-derived land surface temperature (LST) provides global coverage, and is generally utilized to estimate T-a due to the close relationship between LST and T-a. However, LST products are sensitive to cloud contamination, resulting in missing values in LST and leading to the estimated T-a being spatially incomplete. To solve the missing data problem, we propose a deep learning method to estimate spatially seamless T-a from LST that contains missing values. Experimental results on 5-year data of mainland China illustrate that the image-to-image training strategy alleviates the missing data problem and fills the gaps in LST implicitly. Plus, the strong linear relationships between observed daily mean T-a (T-mean), daily minimum T-a (T-min), and daily maximum T-a (T-max) make the estimation of T-mean, T-min, and T(max )simultaneously possible. For mainland China, the proposed method achieves results with R-2 of 0.962, 0.953, 0.944, mean absolute error (MAE) of 1.793 degrees C, 2.143 degrees C, and 2.125 degrees C, and root-mean-square error (RMSE) of 2.376 degrees C, 2.808 degrees C, and 2.823 degrees C for T-mean, T-min, and T-max, respectively. OPeer reviewe

Unsupervised Domain Adaptation for Multispectral Pedestrian Detection

Multimodal information (e.g., visible and thermal) can generate robust pedestrian detections to facilitate around-the-clock computer vision applications, such as autonomous driving and video surveillance. However, it still remains a crucial challenge to train a reliable detector working well in different multispectral pedestrian datasets without manual annotations. In this paper, we propose a novel unsupervised domain adaptation framework for multispectral pedestrian detection, by iteratively generating pseudo annotations and updating the parameters of our designed multispectral pedestrian detector on target domain. Pseudo annotations are generated using the detector trained on source domain, and then updated by fixing the parameters of detector and minimizing the cross entropy loss without back-propagation. Training labels are generated using the pseudo annotations by considering the characteristics of similarity and complementarity between well-aligned visible and infrared image pairs. The parameters of detector are updated using the generated labels by minimizing our defined multi-detection loss function with back-propagation. The optimal parameters of detector can be obtained after iteratively updating the pseudo annotations and parameters. Experimental results show that our proposed unsupervised multimodal domain adaptation method achieves significantly higher detection performance than the approach without domain adaptation, and is competitive with the supervised multispectral pedestrian detectors

Changes in global climate heterogeneity under the 21st century global warming

Publisher Copyright: © 2021 The Author(s)Variations in climate types are commonly used to describe changes in natural vegetation cover in response to global climate change. However, few attempts have been made to quantify the heterogeneous dynamics of climate types. In this study, based on the Coupled Model Intercomparison Project phase 5 (CMIP5) historical and representative concentration pathway (RCP) runs from 18 global climate models, we used Shannon's Diversity Index (SHDI) and Simpson's Diversity Index (SIDI) to characterise of global climate heterogeneity from a morphological perspective. Our results show that global climate heterogeneity calculated by the SHDI/SIDI indices decreased from 1901 to 2095 at a significance level of 0.01. As radiative forcing intensified from RCP 2.6 to 8.5, the SHDI/SIDI decreased significantly. Furthermore, we observed that the spatial distribution of global climate heterogeneity was significantly reduced, with a pronounced latitudinal trend. Sensitivity analysis indicated that the temperature increase played a more significant role in reducing global climate heterogeneity than precipitation under the three warming scenarios, which is possibly attributed to anthropogenic forcing. Our findings suggest that the dynamics of global climate heterogeneity can be an effective means of quantifying global biodiversity loss.Peer reviewe

Observed Changes of Koppen Climate Zones Based on High-Resolution Data Sets in the Qinghai-Tibet Plateau

Emerging and disappearing climate zones are frequently used to diagnose and project climate change. However, little attempt has been made to quantify shifts of climate zones in Qinghai-Tibet Plateau (QTP) based on the high-resolution data sets. Our results show that highland climate was decreased substantially during 1961–2011 and were mainly replaced by boreal climate. We also found that the mean elevation of boreal and highland climate continues to rise, with obvious longitudinal geographical characteristics over the study period. Furthermore, we found that the climate spaces (a climate space defined as the volume of 10°C × 500 mm here) of both boreal and highland climate types tend to be warm and humid ones, which may provide more suitable climate conditions for species to maintain and promote diversity. Characterization of changes in QTP climate types deepens our understanding of regional climate and its biological impacts.Emerging and disappearing climate zones are frequently used to diagnose and project climate change. However, little attempt has been made to quantify shifts of climate zones in Qinghai-Tibet Plateau (QTP) based on the high-resolution data sets. Our results show that highland climate was decreased substantially during 1961-2011 and were mainly replaced by boreal climate. We also found that the mean elevation of boreal and highland climate continues to rise, with obvious longitudinal geographical characteristics over the study period. Furthermore, we found that the climate spaces (a climate space defined as the volume of 10 degrees C x 500 mm here) of both boreal and highland climate types tend to be warm and humid ones, which may provide more suitable climate conditions for species to maintain and promote diversity. Characterization of changes in QTP climate types deepens our understanding of regional climate and its biological impacts. Plain Language Summary Climate classification is the key to simplifying complex climate and helps to deepen the understanding of regional climate change. Based on the high-resolution data set (LZ0025), the sharp climatic gradient features and their potential biological impact on Qinghai-Tibet Plateau (QTP) was quantified. With the temperature increase, the spatial distribution of highland tundra climate was gradually replaced by boreal climate. More importantly, the contraction of highland climate and the expansion of boreal climate has obvious elevation characteristics. In addition, climate spaces of highland and boreal climate types tend to warm and humid ones, which may provide more climatic niches for different species and contribute to regional biodiversity.Peer reviewe

From a Spatial Structure Perspective : Spatial-Temporal Variation of Climate Redistribution of China Based on the Köppen–Geiger Classification

https://doi.org/10.1029/2022GL099319Shifting climate zones are widely used to diagnose and predict regional climate change. However, few attempts have been made to measure the spatial redistribution of these climate zones from a spatial structure perspective. We investigated changes in spatial structure of Köppen climate landscape in China between 1963 and 2098 with a landscape aggregation index. Our results reveal an apparent signal from fragmentation to aggregation, accompanied by the intensification of areal dispersion between cold and warm climate types. Our attribution analysis indicates that anthropogenic forcings have a larger influence on changes of spatial structure than natural variation. We also found that topographical heterogeneity is likely to contribute to the regional spatial fragmentation, especially in the Qinghai-Tibet Plateau. However, we also found that the spatial fragmentation will be weakened around the mid-2040s. We argue that biodiversity is likely to be mediated by spatial structure of future climate landscapes in China.Peer reviewe

Intensification of the dispersion of the global climatic landscape and its potential as a new climate change indicator

Increases and decreases in the areas of climatic types have become one of the most important responses to climate warming. However, few attempts have been made to quantify the complementary relationship between different climate types or to further assess changes in the spatial morphology. In this study, we used different observed datasets to reveal a dispersion phenomenon between major global climate types in 1950-2010, which is significantly consistent with the increasing trend of global temperatures. As the standard deviation of the area of major climate zones strengthened in 1950-2010, the global climatic landscape underwent notable changes. Not only did the area change, but the shape of the overall boundary became regular, the aggregation of climatic patches strengthened, and the climatic diversity declined substantially. However, changes in the global climatic landscapes are not at equilibrium with those on the continental scale. Interpreting these climatic morphological indices can deepen our understanding of the redistribution response mechanisms of species to climate change and help predict how they will be impacted by long-term future climate change.Peer reviewe

Vegetation response to climate zone dynamics and its impacts on surface soil water content and albedo in China

Extensive research has focused on the response of vegetation to climate change, including potential mechanisms and resulting impacts. Although many studies have explored the relationship between vegetation and climate change in China, research on spatiotemporal distribution changes of climate regimes using natural vegetation as an indicator is still lacking. Further, limited information is available on the response of vegetation to shifts in China's regional climatic zones. In this study, we applied Mann-Kendall, and correlation analysis to examine the variabilities in temperature, precipitation, surface soil water, normalised difference vegetation index (NDVI), and albedo in China from 1982 to 2012. Our results indicate significant shifts in the distribution of Koppen-Geiger climate classes in China from 12.08% to 18.98% between 1983 and 2012 at a significance level of 0.05 (MK). The percentage areas in the arid and continental zones expanded at a rate of 0.004%/y and 0.12%/y, respectively, while the percentage area in the temperate and alpine zones decreased by -0.05%/y and - 0.07%/y. Sensitivity fitting results between simulated and observed changes identified temperature to be a dominant control on the dynamics of temperate (r(2)= 0.98) and alpine (r(2)= 0.968) zones, while precipitation was the dominant control on the changes of arid (r(2) = 0.856) and continental (r(2) = 0.815) zones. The response of the NDVI to albedo infers a more pronounced radiative response in temperate (r = -0.82, pPeer reviewe

Spatial Aggregation of Global Dry and Wet Patterns Based on the Standard Precipitation Index

Quantifying the spatial integrity and patterns of dry/wet events over land is essential to understand how the local hydrological regime responds to environmental changes. Spatial aggregation changes in dry and wet areas over land have not been studied extensively. Based on a patch-mosaic landscape model, we analyzed spatial aggregation changes at two levels corresponding to landscape design during 1949 and 2018. At the landscape level, the global aggregation degree increased initially and then weakened around 2006. However, the spatial aggregation process between dry and wet patterns was inconsistent. For the dry pattern, spatial aggregation was mainly caused by area decline induced decreases in the patch number. For the wet pattern, spatial aggregation was caused by area enlargement induced decreases in the patch number. At the class level, with increases in the dry/wet magnitude, the correlation between the affected area and aggregation strengthened. Our results provide new insights to understand the spatial processes and future trends of dry/wet patterns over land. We argue that future vulnerability of agriculture and ecosystems to drought is likely to be further mediated by the changes in drought patterns' spatial structure.Peer reviewe