Contrastive Learning with Prompt-derived Virtual Semantic Prototypes for Unsupervised Sentence Embedding

Contrastive learning has become a new paradigm for unsupervised sentence embeddings. Previous studies focus on instance-wise contrastive learning, attempting to construct positive pairs with textual data augmentation. In this paper, we propose a novel Contrastive learning method with Prompt-derived Virtual semantic Prototypes (ConPVP). Specifically, with the help of prompts, we construct virtual semantic prototypes to each instance, and derive negative prototypes by using the negative form of the prompts. Using a prototypical contrastive loss, we enforce the anchor sentence embedding to be close to its corresponding semantic prototypes, and far apart from the negative prototypes as well as the prototypes of other sentences. Extensive experimental results on semantic textual similarity, transfer, and clustering tasks demonstrate the effectiveness of our proposed model compared to strong baselines. Code is available at https://github.com/lemon0830/promptCSE.Comment: Findings of EMNLP 202

A New Species of Genus Microhyla (Amphibia: Anura: Microhylidae) from Zhejiang Province, China

We described a new species, Microhyla beilunensis sp. nov., from Zhejiang Province of China. Phylogenetic analyses based on the mitochondrial 12S, 16S and CO1 gene sequences suggested that the new taxon was distinctly separated from its congeners and closed to M. mixtura and M. okinavensis. Morphologically, the new species could be identified from its congeners except M. mixtura by several characters: (1) rudimentary webs on toe base; (2) absence of disks and dorsal median longitudinal grooves on finger tips; (3) presence of disks and dorsal median longitudinal grooves on toe tips. As well, the new species could be identified from topotype M. mixtura by the combination of characters: (1) apart from the stripes, bar-shaped and oval-shaped patterns, the rounded spots present on the dorsum of body and legs; (2) the outer metacarpal tubercles prominently larger than the inner one; (3) of males, the ratios of HW, IND, UEW and LAW to SVL of the new species were significantly larger than those of M. mixtura (P < 0.01), and the ratios of SL, IOD, LAHL, HLL, TL, TFL and FL to SVL of the new species were significantly less than those of M. mixtura (P < 0.05)

RecycleGPT: An Autoregressive Language Model with Recyclable Module

Existing large language models have to run K times to generate a sequence of K tokens. In this paper, we present RecycleGPT, a generative language model with fast decoding speed by recycling pre-generated model states without running the whole model in multiple steps. Our approach relies on the observation that adjacent tokens in a sequence usually have strong correlations and the next token in a sequence can be reasonably guessed or inferred based on the preceding ones. Experiments and analysis demonstrate the effectiveness of our approach in lowering inference latency, achieving up to 1.4x speedup while preserving high performance.Comment: Technical Repor

Can the digital economy promote the development of the energy economy? Evidence from China

In this paper, 22 indexes are selected at three levels, including the informatization development level, the Internet development level, and the digital transaction development level, based on China’s provincial panel data from 2011 to 2020, so as to build a digital economy development index system. Moreover, 28 basic indexes are selected from three aspects, including energy construction, energy production and energy consumption, so as to develop an energy economy development evaluation index system. The development index of China’s digital economy and energy economy are measured by using the entropy weight method. The effect of the digital economy on the energy economy and its mechanism are tested by the static panel, the dynamic panel, and the mediating effect and regulating effect models. The results indicate that the digital economy has pronouncedly promoted the development of China’s energy economy, and the development of the digital economy can have an effect on the rationalization of the industrial structure and then affect the development of the energy economy, and there is an intermediary effect. Moreover, the upgrading of the industrial structure is conducive to regulating the digital economy and facilitates the development of the energy economy. The development of the energy economy can be better promoted by focusing on the coordinated regional layout of the digital economy development, building a reliable energy commodity trading platform, and expediting the optimization and upgrading of the industrial structure

Soft Language Clustering for Multilingual Model Pre-training

Multilingual pre-trained language models have demonstrated impressive (zero-shot) cross-lingual transfer abilities, however, their performance is hindered when the target language has distant typology from source languages or when pre-training data is limited in size. In this paper, we propose XLM-P, which contextually retrieves prompts as flexible guidance for encoding instances conditionally. Our XLM-P enables (1) lightweight modeling of language-invariant and language-specific knowledge across languages, and (2) easy integration with other multilingual pre-training methods. On the tasks of XTREME including text classification, sequence labeling, question answering, and sentence retrieval, both base- and large-size language models pre-trained with our proposed method exhibit consistent performance improvement. Furthermore, it provides substantial advantages for low-resource languages in unsupervised sentence retrieval and for target languages that differ greatly from the source language in cross-lingual transfer