Improve Transformer Pre-Training with Decoupled Directional Relative Position Encoding and Representation Differentiations

In this work, we revisit the Transformer-based pre-trained language models and identify two problems that may limit the expressiveness of the model. Firstly, existing relative position encoding models (e.g., T5 and DEBERTA) confuse two heterogeneous information: relative distance and direction. It may make the model unable to capture the associative semantics of the same direction or the same distance, which in turn affects the performance of downstream tasks. Secondly, we notice the pre-trained BERT with Mask Language Modeling (MLM) pre-training objective outputs similar token representations and attention weights of different heads, which may impose difficulties in capturing discriminative semantic representations. Motivated by the above investigation, we propose two novel techniques to improve pre-trained language models: Decoupled Directional Relative Position (DDRP) encoding and MTH pre-training objective. DDRP decouples the relative distance features and the directional features in classical relative position encoding for better position information understanding. MTH designs two novel auxiliary losses besides MLM to enlarge the dissimilarities between (a) last hidden states of different tokens, and (b) attention weights of different heads, alleviating homogenization and anisotropic problem in representation learning for better optimization. Extensive experiments and ablation studies on GLUE benchmark demonstrate the effectiveness of our proposed methods

Mastering Complex Control in MOBA Games with Deep Reinforcement Learning

We study the reinforcement learning problem of complex action control in the Multi-player Online Battle Arena (MOBA) 1v1 games. This problem involves far more complicated state and action spaces than those of traditional 1v1 games, such as Go and Atari series, which makes it very difficult to search any policies with human-level performance. In this paper, we present a deep reinforcement learning framework to tackle this problem from the perspectives of both system and algorithm. Our system is of low coupling and high scalability, which enables efficient explorations at large scale. Our algorithm includes several novel strategies, including control dependency decoupling, action mask, target attention, and dual-clip PPO, with which our proposed actor-critic network can be effectively trained in our system. Tested on the MOBA game Honor of Kings, our AI agent, called Tencent Solo, can defeat top professional human players in full 1v1 games.Comment: AAAI 202

Optimization of a static headspace GC-MS method and its application in metabolic fingerprinting of the leaf volatiles of 42 citrus cultivars

Citrus leaves, which are a rich source of plant volatiles, have the beneficial attributes of rapid growth, large biomass, and availability throughout the year. Establishing the leaf volatile profiles of different citrus genotypes would make a valuable contribution to citrus species identification and chemotaxonomic studies. In this study, we developed an efficient and convenient static headspace (HS) sampling technique combined with gas chromatography-mass spectrometry (GC-MS) analysis and optimized the extraction conditions (a 15-min incubation at 100 ˚C without the addition of salt). Using a large set of 42 citrus cultivars, we validated the applicability of the optimized HS-GC-MS system in determining leaf volatile profiles. A total of 83 volatile metabolites, including monoterpene hydrocarbons, alcohols, sesquiterpene hydrocarbons, aldehydes, monoterpenoids, esters, and ketones were identified and quantified. Multivariate statistical analysis and hierarchical clustering revealed that mandarin (Citrus reticulata Blanco) and orange (Citrus sinensis L. Osbeck) groups exhibited notably differential volatile profiles, and that the mandarin group cultivars were characterized by the complex volatile profiles, thereby indicating the complex nature and diversity of these mandarin cultivars. We also identified those volatile compounds deemed to be the most useful in discriminating amongst citrus cultivars. This method developed in this study provides a rapid, simple, and reliable approach for the extraction and identification of citrus leaf volatile organic compound, and based on this methodology, we propose a leaf volatile profile-based classification model for citrus

Aridity-driven shift in biodiversity–soil multifunctionality relationships

From Springer Nature via Jisc Publications RouterHistory: received 2021-01-07, accepted 2021-08-12, registration 2021-08-25, pub-electronic 2021-09-09, online 2021-09-09, collection 2021-12Publication status: PublishedFunder: National Natural Science Foundation of China (National Science Foundation of China); doi: https://doi.org/10.13039/501100001809; Grant(s): 31770430Abstract: Relationships between biodiversity and multiple ecosystem functions (that is, ecosystem multifunctionality) are context-dependent. Both plant and soil microbial diversity have been reported to regulate ecosystem multifunctionality, but how their relative importance varies along environmental gradients remains poorly understood. Here, we relate plant and microbial diversity to soil multifunctionality across 130 dryland sites along a 4,000 km aridity gradient in northern China. Our results show a strong positive association between plant species richness and soil multifunctionality in less arid regions, whereas microbial diversity, in particular of fungi, is positively associated with multifunctionality in more arid regions. This shift in the relationships between plant or microbial diversity and soil multifunctionality occur at an aridity level of ∼0.8, the boundary between semiarid and arid climates, which is predicted to advance geographically ∼28% by the end of the current century. Our study highlights that biodiversity loss of plants and soil microorganisms may have especially strong consequences under low and high aridity conditions, respectively, which calls for climate-specific biodiversity conservation strategies to mitigate the effects of aridification