Unsupervised Feature Learning by Autoencoder and Prototypical Contrastive Learning for Hyperspectral Classification

Unsupervised learning methods for feature extraction are becoming more and more popular. We combine the popular contrastive learning method (prototypical contrastive learning) and the classic representation learning method (autoencoder) to design an unsupervised feature learning network for hyperspectral classification. Experiments have proved that our two proposed autoencoder networks have good feature learning capabilities by themselves, and the contrastive learning network we designed can better combine the features of the two to learn more representative features. As a result, our method surpasses other comparison methods in the hyperspectral classification experiments, including some supervised methods. Moreover, our method maintains a fast feature extraction speed than baseline methods. In addition, our method reduces the requirements for huge computing resources, separates feature extraction and contrastive learning, and allows more researchers to conduct research and experiments on unsupervised contrastive learning

Corn Yield Prediction based on Remotely Sensed Variables Using Variational Autoencoder and Multiple Instance Regression

In the U.S., corn is the most produced crop and has been an essential part of the American diet. To meet the demand for supply chain management and regional food security, accurate and timely large-scale corn yield prediction is attracting more attention in precision agriculture. Recently, remote sensing technology and machine learning methods have been widely explored for crop yield prediction. Currently, most county-level yield prediction models use county-level mean variables for prediction, ignoring much detailed information. Moreover, inconsistent spatial resolution between crop area and satellite sensors results in mixed pixels, which may decrease the prediction accuracy. Only a few works have addressed the mixed pixels problem in large-scale crop yield prediction. To address the information loss and mixed pixels problem, we developed a variational autoencoder (VAE) based multiple instance regression (MIR) model for large-scaled corn yield prediction. We use all unlabeled data to train a VAE and the well-trained VAE for anomaly detection. As a preprocess method, anomaly detection can help MIR find a better representation of every bag than traditional MIR methods, thus better performing in large-scale corn yield prediction. Our experiments showed that variational autoencoder based multiple instance regression (VAEMIR) outperformed all baseline methods in large-scale corn yield prediction. Though a suitable meta parameter is required, VAEMIR shows excellent potential in feature learning and extraction for large-scale corn yield prediction

Probability-based Global Cross-modal Upsampling for Pansharpening

Pansharpening is an essential preprocessing step for remote sensing image processing. Although deep learning (DL) approaches performed well on this task, current upsampling methods used in these approaches only utilize the local information of each pixel in the low-resolution multispectral (LRMS) image while neglecting to exploit its global information as well as the cross-modal information of the guiding panchromatic (PAN) image, which limits their performance improvement. To address this issue, this paper develops a novel probability-based global cross-modal upsampling (PGCU) method for pan-sharpening. Precisely, we first formulate the PGCU method from a probabilistic perspective and then design an efficient network module to implement it by fully utilizing the information mentioned above while simultaneously considering the channel specificity. The PGCU module consists of three blocks, i.e., information extraction (IE), distribution and expectation estimation (DEE), and fine adjustment (FA). Extensive experiments verify the superiority of the PGCU method compared with other popular upsampling methods. Additionally, experiments also show that the PGCU module can help improve the performance of existing SOTA deep learning pansharpening methods. The codes are available at https://github.com/Zeyu-Zhu/PGCU.Comment: 10 pages, 5 figure

Use of an Active Canopy Sensor Mounted on an Unmanned Aerial Vehicle to Monitor the Growth and Nitrogen Status of Winter Wheat

Using remote sensing to rapidly acquire large-area crop growth information (e.g., shoot biomass, nitrogen status) is an urgent demand for modern crop production; unmanned aerial vehicle (UAV) acts as an effective monitoring platform. In order to improve the practicability and efficiency of UAV based monitoring technique, four field experiments involving different nitrogen (N) rates (0–360 kg N ha−1 ) and seven winter wheat (Triticum aestivum L.) varieties were conducted at different eco-sites (Sihong, Rugao, and Xinghua) during 2015–2019. A multispectral active canopy sensor (RapidSCAN CS-45; Holland Scientific Inc., Lincoln, NE, USA) mounted on a multirotor UAV platform was used to collect the canopy spectral reflectance data of winter wheat at key growth stages, three growth parameters (leaf area index (LAI), leaf dry matter (LDM), plant dry matter (PDM)) and three N indicators (leaf N accumulation (LNA), plant N accumulation (PNA) and N nutrition index (NNI)) were measured synchronously. The quantitative linear relationships between spectral data and six growth indices were systematically analyzed. For monitoring growth and N nutrition status at Feekes stages 6.0–10.0, 10.3–11.1 or entire growth stages, red edge ratio vegetation index (RERVI), red edge chlorophyll index (CIRE) and difference vegetation index (DVI) performed the best among the red edge band-based and red-based vegetation indices, respectively. Across all growth stages, DVI was highly correlated with LAI (R2 = 0.78), LDM (R2 = 0.61), PDM (R2 = 0.63), LNA (R2 = 0.65) and PNA (R2 = 0.73), whereas the relationships between RERVI (R2 = 0.62), CIRE (R2 = 0.62) and NNI had high coefficients of determination. The developed models performed better in monitoring growth indices and N status at Feekes stages 10.3–11.1 than Feekes stages 6.0–10.0. To sum it up, the UAV-mounted active sensor system is able to rapidly monitor the growth and N nutrition status of winter wheat and can be deployed for UAV-based remote-sensing of crops

AdaFuse: Adaptive Medical Image Fusion Based on Spatial-Frequential Cross Attention

Multi-modal medical image fusion is essential for the precise clinical diagnosis and surgical navigation since it can merge the complementary information in multi-modalities into a single image. The quality of the fused image depends on the extracted single modality features as well as the fusion rules for multi-modal information. Existing deep learning-based fusion methods can fully exploit the semantic features of each modality, they cannot distinguish the effective low and high frequency information of each modality and fuse them adaptively. To address this issue, we propose AdaFuse, in which multimodal image information is fused adaptively through frequency-guided attention mechanism based on Fourier transform. Specifically, we propose the cross-attention fusion (CAF) block, which adaptively fuses features of two modalities in the spatial and frequency domains by exchanging key and query values, and then calculates the cross-attention scores between the spatial and frequency features to further guide the spatial-frequential information fusion. The CAF block enhances the high-frequency features of the different modalities so that the details in the fused images can be retained. Moreover, we design a novel loss function composed of structure loss and content loss to preserve both low and high frequency information. Extensive comparison experiments on several datasets demonstrate that the proposed method outperforms state-of-the-art methods in terms of both visual quality and quantitative metrics. The ablation experiments also validate the effectiveness of the proposed loss and fusion strategy

TORE: Token Reduction for Efficient Human Mesh Recovery with Transformer

In this paper, we introduce a set of effective TOken REduction (TORE) strategies for Transformer-based Human Mesh Recovery from monocular images. Current SOTA performance is achieved by Transformer-based structures. However, they suffer from high model complexity and computation cost caused by redundant tokens. We propose token reduction strategies based on two important aspects, i.e., the 3D geometry structure and 2D image feature, where we hierarchically recover the mesh geometry with priors from body structure and conduct token clustering to pass fewer but more discriminative image feature tokens to the Transformer. As a result, our method vastly reduces the number of tokens involved in high-complexity interactions in the Transformer, achieving competitive accuracy of shape recovery at a significantly reduced computational cost. We conduct extensive experiments across a wide range of benchmarks to validate the proposed method and further demonstrate the generalizability of our method on hand mesh recovery. Our code will be publicly available once the paper is published

Analysis of Temporal and Spatial Evolution Characteristics of Land Subsidence in Western Songnen Plain Using Multisource Remote Sensing

AbstractThe exploitation of underground fluid is an important factor leading to land subsidence. The effects of mining depth, frequency, and mode on land subsidence are also different. The objective of this study was to develop a multisource method—including optical remote sensing interpretation, Interferometric Synthetic Aperture Radar (InSAR) technology, and unmanned aerial vehicle (UAV)—to reveal the long-term temporal and spatial evolution law of subsidence characteristics driven by groundwater and oil extraction, as well as to reveal the formation mechanism and seasonal response law of land subsidence under the action of different driving factors. In this paper, we select the western region of Jilin Province located in Songnen Plain as the study area. The subsidence funnels in the study area are distributed in a porphyritic manner, and the distribution of the subsidence funnels has a certain correlation with the distribution of the pumping wells. In farmland areas, the subsidence is mainly caused by pumping groundwater. The annual land subsidence rate in the study area is -3.14 mm/a, and the maximum deformation rate in the study area is -22.05 mm/a. The subsidence is affected by the season, shown by the fact that it rises in the dry season and decreases in the rainy season. The subsidence in the west of Songnen Plain is caused by oil pumping and groundwater pumping, and groundwater pumping is dominant. The exploitation of underground fluid transfers the pressure borne by water or oil to the soil skeleton so as to increase and consolidate the effective stress of the soil layer and lead to land subsidence. The continuous observation of the surface in the western area of Songnen Plain is helpful to guide the safe production of agriculture and industry and ensure the smooth development of local industry and agriculture