On the Difference of BERT-style and CLIP-style Text Encoders

Masked language modeling (MLM) has been one of the most popular pretraining recipes in natural language processing, e.g., BERT, one of the representative models. Recently, contrastive language-image pretraining (CLIP) has also attracted attention, especially its vision models that achieve excellent performance on a broad range of vision tasks. However, few studies are dedicated to studying the text encoders learned by CLIP. In this paper, we analyze the difference between BERT-style and CLIP-style text encoders from three experiments: (i) general text understanding, (ii) vision-centric text understanding, and (iii) text-to-image generation. Experimental analyses show that although CLIP-style text encoders underperform BERT-style ones for general text understanding tasks, they are equipped with a unique ability, i.e., synesthesia, for the cross-modal association, which is more similar to the senses of humans.Comment: Natural Language Processing. 10 pages, 1 figure. Findings of ACL-202

CMB: A Comprehensive Medical Benchmark in Chinese

Large Language Models (LLMs) provide a possibility to make a great breakthrough in medicine. The establishment of a standardized medical benchmark becomes a fundamental cornerstone to measure progression. However, medical environments in different regions have their local characteristics, e.g., the ubiquity and significance of traditional Chinese medicine within China. Therefore, merely translating English-based medical evaluation may result in \textit{contextual incongruities} to a local region. To solve the issue, we propose a localized medical benchmark called CMB, a Comprehensive Medical Benchmark in Chinese, designed and rooted entirely within the native Chinese linguistic and cultural framework. While traditional Chinese medicine is integral to this evaluation, it does not constitute its entirety. Using this benchmark, we have evaluated several prominent large-scale LLMs, including ChatGPT, GPT-4, dedicated Chinese LLMs, and LLMs specialized in the medical domain. It is worth noting that our benchmark is not devised as a leaderboard competition but as an instrument for self-assessment of model advancements. We hope this benchmark could facilitate the widespread adoption and enhancement of medical LLMs within China. Check details in \url{https://cmedbenchmark.llmzoo.com/}

HuatuoGPT, towards Taming Language Model to Be a Doctor

In this paper, we present HuatuoGPT, a large language model (LLM) for medical consultation. The core recipe of HuatuoGPT is to leverage both \textit{distilled data from ChatGPT} and \textit{real-world data from doctors} in the supervised fine-tuned stage. The responses of ChatGPT are usually detailed, well-presented and informative while it cannot perform like a doctor in many aspects, e.g. for integrative diagnosis. We argue that real-world data from doctors would be complementary to distilled data in the sense the former could tame a distilled language model to perform like doctors. To better leverage the strengths of both data, we train a reward model to align the language model with the merits that both data bring, following an RLAIF (reinforced learning from AI feedback) fashion. To evaluate and benchmark the models, we propose a comprehensive evaluation scheme (including automatic and manual metrics). Experimental results demonstrate that HuatuoGPT achieves state-of-the-art results in performing medical consultation among open-source LLMs in GPT-4 evaluation, human evaluation, and medical benchmark datasets. It is worth noting that by using additional real-world data and RLAIF, the distilled language model (i.e., HuatuoGPT) outperforms its teacher model ChatGPT in most cases. Our code, data, and models are publicly available at \url{https://github.com/FreedomIntelligence/HuatuoGPT}. The online demo is available at \url{https://www.HuatuoGPT.cn/}

Quantitatively analyzing the failure processes of rechargeable Li metal batteries.

Practical use of lithium (Li) metal for high–energy density lithium metal batteries has been prevented by the continuous formation of Li dendrites, electrochemically isolated Li metal, and the irreversible formation of solid electrolyte interphases (SEIs). Differentiating and quantifying these inactive Li species are key to understand the failure mode. Here, using operando nuclear magnetic resonance (NMR) spectroscopy together with ex situ titration gas chromatography (TGC) and mass spectrometry titration (MST) techniques, we established a solid foundation for quantifying the evolution of dead Li metal and SEI separately. The existence of LiH is identified, which causes deviation in the quantification results of dead Li metal obtained by these three techniques. The formation of inactive Li under various operating conditions has been studied quantitatively, which revealed a general “two-stage” failure process for the Li metal. The combined techniques presented here establish a benchmark to unravel the complex failure mechanism of Li metal

Additives synergy for stable interface formation on rechargeable lithium metal anodes

Abstract(#br)The attention towards lithium (Li) metal anodes has been rekindled in recent years as it would boost the energy-density of Li batteries. However, notorious safety issues and cycling instability severely hinder their commercialization, especially when cycled in traditional carbonic ester electrolytes that exhibit a wide voltage window and are compatible with most of the cathode materials. Herein, lithium difluorophosphate (LiDFP) and vinylene carbonate (VC) are combined, and demonstrated to be synergistic in constructing in situ a mechanically stable and highly Li-ion conducting surface film on the Li metal anode. This results in uniform and compact Li deposition largely suppressing the formation of Li dendrites, dead lithium and irreversible Li-species as revealed by operando neutron depth profiling (NDP). This enables long-term cycling stability and enhancement of the Coulombic efficiency for rechargeable Li metal anodes. By combining solid state nuclear magnetic resonance (SSNMR) and spectroscopic studies, it is demonstrated that VC slows down the LiDFP reduction, yet promoting the breaking of the P–F bonds, which leads to a protective film. This film is rich in LiF–Li 3 PO 4 inorganic compounds, distributed homogeneously, that is embedded in a matrix of P–O–C species and macromolecular organic compounds like lithium ethylene dicarbonate. This composition is responsible for the improved ionic conductivity and mechanical stability of the protective film during extended cycles. The detailed insight in the additives interaction provides new opportunities for the design of rational surface films necessary for realizing high-performance lithium metal batteries

P2-Na0.67 Alx Mn1-x O2 : Cost-Effective, Stable and High-Rate Sodium Electrodes by Suppressing Phase Transitions and Enhancing Sodium Cation Mobility.

Sodium layered P2-stacking Na0.67 MnO2 materials have shown great promise for sodium-ion batteries. However, the undesired Jahn-Teller effect of the Mn4+ /Mn3+ redox couple and multiple biphasic structural transitions during charge/discharge of the materials lead to anisotropic structure expansion and rapid capacity decay. Herein, by introducing abundant Al into the transition-metal layers to decrease the number of Mn3+ , we obtain the low cost pure P2-type Na0.67 Alx Mn1-x O2 (x=0.05, 0.1 and 0.2) materials with high structural stability and promising performance. The Al-doping effect on the long/short range structural evolutions and electrochemical performances is further investigated by combining in situ synchrotron XRD and solid-state NMR techniques. Our results reveal that Al-doping alleviates the phase transformations thus giving rise to better cycling life, and leads to a larger spacing of Na+ layer thus producing a remarkable rate capability of 96 mAh g-1 at 1200 mA g-1

Unraveling (electro)-chemical stability and interfacial reactions of Li 10 SnP 2 S 12 in all-solid-state Li batteries

Abstract(#br)Li 10 SnP 2 S 12 (LSPS) with high ionic conductivity and moderate price is a promising solid electrolyte for all-solid-state batteries. However, the instability of LSPS and LSPS/electrodes interfaces would cause poor cycle performance issues in the LSPS-based all-solid-state batteries, which have not been well understood. Herein, we address and unravel the decomposition products of LSPS and their Li + transfer characteristics, especially on the surface of LSPS/electrodes by using solid-state nuclear magnetic resonance (ss NMR) spectroscopy coupled with X-ray photoelectron spectroscopy (XPS). The results reveal that the high mechanical energy during ball-milling process leads to the decomposition of LSPS into Li 4 SnS 4 and Li 3 PS 4 . During charge/discharge cycling, specific capacity fading of batteries originates from the formation of new interfacial layer at LSPS/Acetylene black cathode and LSPS/Li metal anode interfaces. Furthermore, our results demonstrate that the rough and porous morphology of the interface formed after cycling, rather than the decomposition products, is the critical factor which results in the increases of the interfacial resistance at LSPS/Li interface and serious formation of Li dendrite. Our results highlight the significant roles of (electro)chemical and interfacial stability of sulfide solid electrolyte in the development of all-solid-state batteries

Identification of Endogenous Controls for Analyzing Serum Exosomal miRNA in Patients with Hepatitis B or Hepatocellular Carcinoma

Serum exosomal microRNAs (miRNAs) have received considerable attention as potential biomarkers for diagnosing cancer. The canonical technique for measuring miRNA transcript levels is reverse transcription quantitative polymerase chain reaction (RT-qPCR). One prerequisite for validating RT-qPCR data is proper normalization with respect to stably expressed endogenous reference genes. However, genes that meet all of the criteria of a control gene for exosomal miRNAs have not yet been identified. To find out the control gene for exosomal miRNAs, we evaluated the expression stability of 11 well-known reference genes in circulating exosomes. In this study, we found that the combination of miR-221, miR-191, let-7a, miR-181a, and miR-26a can be an optimal gene reference set for normalizing the expression of liver-specific miRNAs. This combination enhanced the robustness of the relative quantification analyses. These findings highlight the importance of validating reference genes before quantifying target miRNAs. Furthermore, our findings will improve studies that monitor hepatitis progression and will aid in the discovery of noninvasive biomarkers to diagnose early stage HCC

Prevalence of Chlamydia trachomatis, Neisseria gonorrhoeae, Mycoplasma genitalium and Ureaplasma urealyticum infections using a novel isothermal simultaneous RNA amplification testing method in infertile males

Abstract Background The purpose of this study was to evaluate the prevalence of Chlamydia trachomatis, Neisseria gonorrhoeae, Mycoplasma genitalium and Ureaplasma urealyticum infections in infertile men that consulted our outpatient departments using a novel simultaneous amplification testing (SAT) that is RNA-detection based. The possible impact of C. trachomatis, N. gonorrhoeae, M. genitalium and U. urealyticum infections on semen parameters was also noted in the present study. Methods A total of 2607 males that were diagnosed with infertility were included in this study. C. trachomatis, N. gonorrhoeae, M. genitalium and U. urealyticum infections were detected in the urine samples using SAT method. Related data, including semen parameters and age as well as C. trachomatis, N. gonorrhoeae, M. genitalium and U. urealyticum infections were collected and analyzed. Results A total of 51 and 1418 urine samples were found positive for M. genitalium RNA and U. urealyticum RNA, respectively, while the prevalence of C. trachomatis and N. gonorrhoeae was relatively lower. Men with positive M. genitalium RNA and U. urealyticum RNA had higher sperm DNA fragmentation index (DFI) while the comparisons of other semen parameters yielded nonsignificant results between the RNA positive and negative group. A multivariate linear regression analysis revealed that U. urealyticum and M. genitalium infections posed significant factors of DFI (adjusted R2 = 46.2%). Conclusions Our study suggested a relative high prevalence of U. urealyticum and M. genitalium infection based on this novel SAT detection method. U. urealyticum and M. genitalium infection could possibly impair male fertility potential through promoting sperm DNA damage

Investigation of the fungal community structures of imported wheat using high-throughput sequencing technology.

This study introduced the application of high-throughput sequencing techniques to the investigation of microbial diversity in the field of plant quarantine. It examined the microbial diversity of wheat imported into China, and established a bioinformatics database of wheat pathogens based on high-throughput sequencing results. This study analyzed the nuclear ribosomal internal transcribed spacer (ITS) region of fungi through Illumina Miseq sequencing to investigate the fungal communities of both seeds and sieve-through. A total of 758,129 fungal ITS sequences were obtained from ten samples collected from five batches of wheat imported from the USA. These sequences were classified into 2 different phyla, 15 classes, 33 orders, 41 families, or 78 genera, suggesting a high fungal diversity across samples. Apairwise analysis revealed that the diversity of the fungal community in the sieve-through is significantly higher than those in the seeds. Taxonomic analysis showed that at the class level, Dothideomycetes dominated in the seeds and Sordariomycetes dominated in the sieve-through. In all, this study revealed the fungal community composition in the seeds and sieve-through of the wheat, and identified key differences in the fungal community between the seeds and sieve-through