Generate to Understand for Representation

In recent years, a significant number of high-quality pretrained models have emerged, greatly impacting Natural Language Understanding (NLU), Natural Language Generation (NLG), and Text Representation tasks. Traditionally, these models are pretrained on custom domain corpora and finetuned for specific tasks, resulting in high costs related to GPU usage and labor. Unfortunately, recent trends in language modeling have shifted towards enhancing performance through scaling, further exacerbating the associated costs. Introducing GUR: a pretraining framework that combines language modeling and contrastive learning objectives in a single training step. We select similar text pairs based on their Longest Common Substring (LCS) from raw unlabeled documents and train the model using masked language modeling and unsupervised contrastive learning. The resulting model, GUR, achieves impressive results without any labeled training data, outperforming all other pretrained baselines as a retriever at the recall benchmark in a zero-shot setting. Additionally, GUR maintains its language modeling ability, as demonstrated in our ablation experiment. Our code is available at \url{https://github.com/laohur/GUR}

The Influence of Higher Education on China’s Macro-Economy

Education contributes to economic growth. China’s economy has shifted from high growth to higher-quality development, creating a new demand to reform and develop higher education. This paper first measures the composite index of higher education in China from 2009 to 2019 by combining the Cobb-Douglas production function and Denison’s economic growth factors analysis method. Results showed that higher education in China has an increasing contribution rate to economic growth from 2009 to 2019 but is still lagging behind developed countries. Second, it analyzes the new trends of higher education development in developed countries, compares the development models of higher education in different countries, and reveals the problems in China’s higher education. Finally, it examines the impact of higher education on China’s macro-economy from the human capital theory and the economies of scale theory. Furthermore, it proposes strategies for the higher-quality development of higher education and provides a reference for adjusting and optimizing the development model of higher education

Studies on the formation and forming mechanism of the related substance E in potassium clavulanate production by HPLC-MS/MS

O objetivo deste estudo foi investigar a formação da substância E e o respectivo mecanismo na produção de clavulanato de potássio. Confirmou-se a impureza com tempo de retenção de 11,1 min no produto final, clavulanato de potássio, como substância E, por meio de cromatografia líquida de alta eficiência, em conjunto com detecção por espectrometria de massas (CLAE-MS-MS). A análise da substância relacionada E durante a produção do ácido clavulânico mostrou que essa impureza pode ser formada tanto durante a fermentação quanto durante os processos de purificação, especialmente no estágio final de fermentação, filtração, concentração e procedimento de extração. O ácido clavulãnico foi o precursor da substância E. Estudos no mecanismo de sua formação mostraram que a substância E formou-se pela combinação do grupo imina da molécula do ácido clavulânico com o grupo carboxílico de outra molécula de ácido clavulânico, com a abertura do anel β-lactâmico. Resultados do teste ortogonal multifatorial confirmaram que a concentração do ácido clavulânico foi o fator dominante para acelerar a reação, enquanto a temperatura foi outro fator que contribuiu. O pH de 5,0 a 6,5 teve pouco impacto na geração da substância E.The objective of this study was to investigate the formation and forming mechanism of the related substance E in potassium clavulanate production. The impurity with retention time of 11.1 min in potassium clavulanate final product was confirmed as the related substance E by high performance liquid chromatography with tandem mass spectrometric detection (HPLC-MS/MS).The related substance E analysis during the production of clavulanic acid showed that this impurity could be formed during both the fermentation and purification processes, especially in the later fermentation stage, filtration concentration and back-extraction procedure. Clavulanic acid was the precursor of the related substance E. Studies on its forming mechanism showed that the related substance E was formed by the combination of the imino group of one molecule of clavulanic acid with the carboxyl group of another molecule of clavulanic acid with the opening of β-lactam ring. Results of a multi-factor orthogonal test confirmed that the concentration of clavulanic acid was the dominant factor to accelerate the reaction, while the temperature was another contributing factor. The pH 5.0-6.5 had little impact on the generation of the related substance E

Application of Seasonal CRM Integrations to Develop Statistics and Improved GCM Parameterization of Subgrid Cloud-Radiation Interactions

The works supported by this ARM project lay the solid foundation for improving the parameterization of subgrid cloud-radiation interactions in the NCAR CCSM and the climate simulations. We have made a significant use of CRM simulations and concurrent ARM observations to produce long-term, consistent cloud and radiative property datasets at the cloud scale (Wu et al. 2006, 2007). With these datasets, we have investigated the mesoscale enhancement of cloud systems on surface heat fluxes (Wu and Guimond 2006), quantified the effects of cloud horizontal inhomogeneity and vertical overlap on the domain-averaged radiative fluxes (Wu and Liang 2005), and subsequently validated and improved the physically-based mosaic treatment of subgrid cloud-radiation interactions (Liang and Wu 2005). We have implemented the mosaic treatment into the CCM3. The 5-year (1979-1983) AMIP-type simulation showed significant impacts of subgrid cloud-radiation interaction on the climate simulations (Wu and Liang 2005). We have actively participated in CRM intercomparisons that foster the identification and physical understanding of common errors in cloud-scale modeling (Xie et al. 2005; Xu et al. 2005, Grabowski et al. 2005)