Towards Effective Context for Meta-Reinforcement Learning: an Approach based on Contrastive Learning

Context, the embedding of previous collected trajectories, is a powerful construct for Meta-Reinforcement Learning (Meta-RL) algorithms. By conditioning on an effective context, Meta-RL policies can easily generalize to new tasks within a few adaptation steps. We argue that improving the quality of context involves answering two questions: 1. How to train a compact and sufficient encoder that can embed the task-specific information contained in prior trajectories? 2. How to collect informative trajectories of which the corresponding context reflects the specification of tasks? To this end, we propose a novel Meta-RL framework called CCM (Contrastive learning augmented Context-based Meta-RL). We first focus on the contrastive nature behind different tasks and leverage it to train a compact and sufficient context encoder. Further, we train a separate exploration policy and theoretically derive a new information-gain-based objective which aims to collect informative trajectories in a few steps. Empirically, we evaluate our approaches on common benchmarks as well as several complex sparse-reward environments. The experimental results show that CCM outperforms state-of-the-art algorithms by addressing previously mentioned problems respectively.Comment: Accepted to AAAI 202

Comparison of Different Transfer Learning Methods for Classification of Mangrove Communities Using MCCUNet and UAV Multispectral Images

Mangrove-forest classification by using deep learning algorithms has attracted increasing attention but remains challenging. The current studies on the transfer classification of mangrove communities between different regions and different sensors are especially still unclear. To fill the research gap, this study developed a new deep-learning algorithm (encoder–decoder with mixed depth-wise convolution and cascade upsampling, MCCUNet) by modifying the encoder and decoder sections of the DeepLabV3+ algorithm and presented three transfer-learning strategies, namely frozen transfer learning (F-TL), fine-tuned transfer learning (Ft-TL), and sensor-and-phase transfer learning (SaP-TL), to classify mangrove communities by using the MCCUNet algorithm and high-resolution UAV multispectral images. This study combined the deep-learning algorithms with recursive feature elimination and principal component analysis (RFE–PCA), using a high-dimensional dataset to map and classify mangrove communities, and evaluated their classification performance. The results of this study showed the following: (1) The MCCUNet algorithm outperformed the original DeepLabV3+ algorithm for classifying mangrove communities, achieving the highest overall classification accuracy (OA), i.e., 97.24%, in all scenarios. (2) The RFE–PCA dimension reduction improved the classification performance of deep-learning algorithms. The OA of mangrove species from using the MCCUNet algorithm was improved by 7.27% after adding dimension-reduced texture features and vegetation indices. (3) The Ft-TL strategy enabled the algorithm to achieve better classification accuracy and stability than the F-TL strategy. The highest improvement in the F1–score of Spartina alterniflora was 19.56%, using the MCCUNet algorithm with the Ft-TL strategy. (4) The SaP-TL strategy produced better transfer-learning classifications of mangrove communities between images of different phases and sensors. The highest improvement in the F1–score of Aegiceras corniculatum was 19.85%, using the MCCUNet algorithm with the SaP-TL strategy. (5) All three transfer-learning strategies achieved high accuracy in classifying mangrove communities, with the mean F1–score of 84.37~95.25%

Huatuo-26M, a Large-scale Chinese Medical QA Dataset

In this paper, we release a largest ever medical Question Answering (QA) dataset with 26 million QA pairs. We benchmark many existing approaches in our dataset in terms of both retrieval and generation. Experimental results show that the existing models perform far lower than expected and the released dataset is still challenging in the pre-trained language model era. Moreover, we also experimentally show the benefit of the proposed dataset in many aspects: (i) trained models for other QA datasets in a zero-shot fashion; and (ii) as external knowledge for retrieval-augmented generation (RAG); and (iii) improving existing pre-trained language models by using the QA pairs as a pre-training corpus in continued training manner. We believe that this dataset will not only contribute to medical research but also facilitate both the patients and clinical doctors. See \url{https://github.com/FreedomIntelligence/Huatuo-26M}

Tropical cyclone footprint in the ocean mixed layer observed by Argo in the Northwest Pacific

The article of record as published may be located at http://dx.doi.org/10.1002/2014JC010316This study systematically investigated the ocean mixed layer responses to tropical cyclone (TC) using available Argo profiles during the period of 1998–2011 in the northwest Pacific. Results reveal that isothermal layer (IL) deepening and isothermal layer (IL) cooling with evident rightward biases induced by strong TCs are clearer compared to the weak TCs. Likewise, the rightward biases of IL deepening and cooling induced by fast TCs are more obvious than that induced by slow TCs. The upwelling within TC’s eye is much stronger for the strong (slow) TCs than weak (fast) TCs. For the strong and slow TCs, the TC-induced rainfall reduces deepening of constant density layer (with its depth called the mixed layer depth, MLD), and in turn increases the barrier layer thickness (BLT). The initial BL prior to TC can restrict IL cooling more markedly under the weak and fast TCs than under the strong and slow TCs. The inertial oscillation is stronger induced by the strong (fast) TCs than by the weak (slow) TCs. In addition, the most pronounced TC-induced mixed layer deepening and IL cooling in July to October climatology occur in the subtropical gyre of the northwest Pacific with enhanced vertical diffusivity. The maximum increase of isothermal layer depth (ILD) and MLD is up to 5 m, with IL cooling up to 0.4 C.This study was supported by the National Basic Research Program of China (2013CB430304), National Natural Science Foundation (41030854, 41106005, 41176003, 41206178, and 41376015) of China, and National High- Tech R&D Program (2013AA09A505) of China. Peter C. Chu was supported by the Office of Naval Research and Naval Oceanographic Office

Multimodal Classification Framework Based on Hypergraph Latent Relation for End-Stage Renal Disease Associated with Mild Cognitive Impairment

Combined arterial spin labeling (ASL) and functional magnetic resonance imaging (fMRI) can reveal more comprehensive properties of the spatiotemporal and quantitative properties of brain networks. Imaging markers of end-stage renal disease associated with mild cognitive impairment (ESRDaMCI) will be sought from these properties. The current multimodal classification methods often neglect to collect high-order relationships of brain regions and remove noise from the feature matrix. A multimodal classification framework is proposed to address this issue using hypergraph latent relation (HLR). A brain functional network with hypergraph structural information is constructed by fMRI data. The feature matrix is obtained through graph theory (GT). The cerebral blood flow (CBF) from ASL is selected as the second modal feature matrix. Then, the adaptive similarity matrix is constructed by learning the latent relation between feature matrices. Latent relation adaptive similarity learning (LRAS) is introduced to multi-task feature learning to construct a multimodal feature selection method based on latent relation (LRMFS). The experimental results show that the best classification accuracy (ACC) reaches 88.67%, at least 2.84% better than the state-of-the-art methods. The proposed framework preserves more valuable information between brain regions and reduces noise among feature matrixes. It provides an essential reference value for ESRDaMCI recognition

TorchOpt

<h2>What's Changed [0.7.3] - 2023-11-10</h2> <h3>Changed</h3> <ul> <li>Set minimal C++ standard to C++17 by @XuehaiPan in #195.</li> </ul> <h3>Fixed</h3> <ul> <li>Fix <code>optree</code> compatibility for multi-tree-map with <code>None</code> values by @XuehaiPan in #195.</li> </ul> <p><strong>Full Changelog</strong>: https://github.com/metaopt/torchopt/compare/v0.7.2...v0.7.3</p>If you use this software, please cite it as below

Effects of Fish Meal Replacement with Fermented Antarctic Krill Meal on Growth Performance, Body Color and Serum Biochemical Indexes of Koi Carp (Cyprinus carpio L.)

To investigate the effects of replacing fish meal with fermented Antarctic krill meal on growth performance, body color, and serum biochemical indexes of koi carp, 450 koi carp with an average body weight of (4.92±0.22) g were randomly divided into five groups with three replicates per group and 30 fish per replicate. In the control group, we added unfermented defatted Antarctic krill meal, and in the experimental groups, we added defatted Antarctic krill meal fermented by Enterococcus faecalis, Lactobacillus plantarum, Clostridium butyricum, and compound bacteria (1:1:1), which were termed as ES (control group), EF, LP, CB, and MIX, respectively. The addition amount of Antarctic krill meal in each group was 200 g/kg, and the experiment lasted for 70 days. The results showed the following: Compared with those in the control group, the replacement of fish meal with fermented Antarctic krill meal significantly improved final body weight (FBW), weight gain rate (WGR), and specific growth rate (SGR) (P0.05). Based on the above results, using complex bacteria to ferment krill meal in the koi carp feed is recommended with a 20% addition level of defatted krill meal, which can effectively improve the absorption and conversion utilization rate of krill meal

Combination of super-resolution reconstruction and SGA-Net for marsh vegetation mapping using multi-resolution multispectral and hyperspectral images

Vegetation is crucial for wetland ecosystems. Human activities and climate changes are increasingly threatening wetland ecosystems. Combining satellite images and deep learning for classifying marsh vegetation communities has faced great challenges because of its coarse spatial resolution and limited spectral bands. This study aimed to propose a method to classify marsh vegetation using multi-resolution multispectral and hyperspectral images, combining super-resolution techniques and a novel self-constructing graph attention neural network (SGA-Net) algorithm. The SGA-Net algorithm includes a decoding layer (SCE-Net) to precisely fine marsh vegetation classification in Honghe National Nature Reserve, Northeast China. The results indicated that the hyperspectral reconstruction images based on the super-resolution convolutional neural network (SRCNN) obtained higher accuracy with a peak signal-to-noise ratio (PSNR) of 28.87 and structural similarity (SSIM) of 0.76 in spatial quality and root mean squared error (RMSE) of 0.11 and R2 of 0.63 in spectral quality. The improvement of classification accuracy (MIoU) by enhanced super-resolution generative adversarial network (ESRGAN) (6.19%) was greater than that of SRCNN (4.33%) and super-resolution generative adversarial network (SRGAN) (3.64%). In most classification schemes, the SGA-Net outperformed DeepLabV3 + and SegFormer algorithms for marsh vegetation and achieved the highest F1-score (78.47%). This study demonstrated that collaborative use of super-resolution reconstruction and deep learning is an effective approach for marsh vegetation mapping

Effects of Multi-Growth Periods UAV Images on Classifying Karst Wetland Vegetation Communities Using Object-Based Optimization Stacking Algorithm

Combining machine learning algorithms with multi-temporal remote sensing data for fine classification of wetland vegetation has received wide attention from researchers. However, wetland vegetation has different physiological characteristics and phenological information in different growth periods, so it is worth exploring how to use different growth period characteristics to achieve fine classification of vegetation communities. To resolve these issues, we developed an ensemble learning model by stacking Random Forest (RF), CatBoost, and XGBoost algorithms for karst wetland vegetation community mapping and evaluated its classification performance using three growth periods of UAV images. We constructed six classification scenarios to quantitatively evaluate the effects of combining multi-growth periods UAV images on identifying vegetation communities in the Huixian Karst Wetland of International Importance. Finally, we clarified the influence and contribution of different feature bands on vegetation communities’ classification from local and global perspectives based on the SHAP (Shapley Additive explanations) method. The results indicated that (1) the overall accuracies of the four algorithms ranged from 82.03% to 93.37%, and the classification performance was Stacking > CatBoost > RF > XGBoost in order. (2) The Stacking algorithm significantly improved the classification results of vegetation communities, especially Huakolasa, Reed-Imperate, Linden-Camphora, and Cephalanthus tetrandrus-Paliurus ramosissimus. Stacking had better classification performance and generalization ability than the other three machine learning algorithms. (3) Our study confirmed that the combination of spring, summer, and autumn growth periods of UAV images produced the highest classification accuracy (OA, 93.37%). In three growth periods, summer-based UAVs achieved the highest classification accuracy (OA, 85.94%), followed by spring (OA, 85.32%) and autumn (OA, 84.47%) growth period images. (4) The interpretation of black-box stacking model outputs found that vegetation indexes and texture features provided more significant contributions to classifying karst wetland vegetation communities than the original spectral bands, geometry features, and position features. The vegetation indexes (COM and NGBDI) and texture features (Homogeneity and Standard Deviation) were very sensitive when distinguishing Bermudagrass, Bamboo, and Linden-Camphora. These research findings provide a scientific basis for the protection, restoration, and sustainable development of karst wetlands