Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model

Pre-trained multilingual language models play an important role in cross-lingual natural language understanding tasks. However, existing methods did not focus on learning the semantic structure of representation, and thus could not optimize their performance. In this paper, we propose Multi-level Multilingual Knowledge Distillation (MMKD), a novel method for improving multilingual language models. Specifically, we employ a teacher-student framework to adopt rich semantic representation knowledge in English BERT. We propose token-, word-, sentence-, and structure-level alignment objectives to encourage multiple levels of consistency between source-target pairs and correlation similarity between teacher and student models. We conduct experiments on cross-lingual evaluation benchmarks including XNLI, PAWS-X, and XQuAD. Experimental results show that MMKD outperforms other baseline models of similar size on XNLI and XQuAD and obtains comparable performance on PAWS-X. Especially, MMKD obtains significant performance gains on low-resource languages.Comment: accepted at EMNLP 202

Prediction model of cervical lymph node metastasis based on clinicopathological characteristics of papillary thyroid carcinoma: a dual-center retrospective study

BackgroundThe overall prevalence of papillary thyroid carcinoma (PTC) patients is expanding along with an ongoing increase in thyroid cancer incidence. Patients with PTC who have lymph node metastases have a poor prognosis and a high death rate. There is an urgent need for indicators that can predict lymph node metastasis (LNM) before surgery as current imaging techniques, such as ultrasonography, do not have sufficient sensitivity to detect LNM. To predict independent risk factors for Central lymph node metastasis (CLNM) or Lateral lymph node metastasis (LLNM), we therefore developed two nomograms based on CLNM and LLNM, separately.MethodsIn two centers, the Second Affiliated Hospital of Nanchang University and Yichun People’s Hospital, we retrospectively analyzed clinicopathological characteristics of PTC patients. We utilized multivariate analysis to screen for variables that might be suspiciously related to CLNM or LLNM. Furthermore, we developed nomograms to graphically depict the independent risk valuables connected to lymph node metastasis in PTC patients.ResultUltimately, 6068 PTC patients in all were included in the research. Six factors, including age<45, male, mETE, TSH>1.418, tumor size>4cm, and location (multicentric and lobe), were observed to be related to CLNM. Age<45, male, mETE (minimal extrathyroidal extension), multifocality, TSH≥2.910, CLNM positive, and tumor size>4cm were regarded as related risk factors for LLNM. The two nomograms developed subsequently proved to have good predictive power with 0.706 and 0.818 and demonstrated good clinical guidance functionality with clinical decision curves and impact curves.ConclusionBased on the successful establishment of this dual-institution-based visual nomogram model, we found that some clinical features are highly correlated with cervical lymph node metastasis, including CLNM and LLNM, which will better help clinicians make individualized clinical decisions for more effectively rationalizing managing PTC patients

Biodegradable Silk Fibroin Nanocarriers to Modulate Hypoxia Tumor Microenvironment Favoring Enhanced Chemotherapy

Biopolymer silk fibroin (SF) is a great candidate for drug carriers characterized by its tunable biodegradability, and excellent biocompatibility properties. Recently, we have constructed SF-based nano-enabled drug delivery carriers, in which doxorubicin (Dox) and atovaquone (Ato) were encapsulated with Arg-Gly-Asp-SF-Polylactic Acid (RSA) to form micellar-like nanoparticles (RSA-Dox-Ato NPs). The RGD peptide was decorated on micellar-like nanoparticles, promoting tumor accumulation of the drug. Meanwhile, Ato, as a mitochondrial complex III inhibitor inhibiting mitochondrial respiration, would reverse the hypoxia microenvironment and enhance chemotherapy in the tumor. In vitro, the biopolymer alone showed extremely low cytotoxicity to 4T1 cell lines, while the RSA-Dox-Ato demonstrated a higher inhibition rate than other groups. Most significantly, the ROS levels in cells were obviously improved after being treated with RSA-Dox-Ato, indicating that the hypoxic microenvironment was alleviated. Eventually, SF-based targeted drug carrier provides biocompatibility to reverse hypoxia microenvironment in vivo for enhancing chemotherapy, strikingly suppressing tumor development, and thereby suggesting a promising candidate for drug delivery system

Two ultraviolet radiation datasets that cover China

Ultraviolet (UV) radiation has significant effects on ecosystems, environments, and human health, as well as atmospheric processes and climate change. Two ultraviolet radiation datasets are described in this paper. One contains hourly observations of UV radiation measured at 40 Chinese Ecosystem Research Network stations from 2005 to 2015. CUV3 broadband radiometers were used to observe the UV radiation, with an accuracy of 5%, which meets the World Meteorology Organization's measurement standards. The extremum method was used to control the quality of the measured datasets. The other dataset contains daily cumulative UV radiation estimates that were calculated using an all-sky estimation model combined with a hybrid model. The reconstructed daily UV radiation data span from 1961 to 2014. The mean absolute bias error and root-mean-square error are smaller than 30% at most stations, and most of the mean bias error values are negative, which indicates underestimation of the UV radiation intensity. These datasets can improve our basic knowledge of the spatial and temporal variations in UV radiation. Additionally, these datasets can be used in studies of potential ozone formation and atmospheric oxidation, as well as simulations of ecological processes

Retrospective evaluation of whole exome and genome mutation calls in 746 cancer samples

Funder: NCI U24CA211006Abstract: The Cancer Genome Atlas (TCGA) and International Cancer Genome Consortium (ICGC) curated consensus somatic mutation calls using whole exome sequencing (WES) and whole genome sequencing (WGS), respectively. Here, as part of the ICGC/TCGA Pan-Cancer Analysis of Whole Genomes (PCAWG) Consortium, which aggregated whole genome sequencing data from 2,658 cancers across 38 tumour types, we compare WES and WGS side-by-side from 746 TCGA samples, finding that ~80% of mutations overlap in covered exonic regions. We estimate that low variant allele fraction (VAF < 15%) and clonal heterogeneity contribute up to 68% of private WGS mutations and 71% of private WES mutations. We observe that ~30% of private WGS mutations trace to mutations identified by a single variant caller in WES consensus efforts. WGS captures both ~50% more variation in exonic regions and un-observed mutations in loci with variable GC-content. Together, our analysis highlights technological divergences between two reproducible somatic variant detection efforts

Relationships between spirituality and mental stress in people living with HIV in China: a cross-sectional study

There are limited reports on the relationship between spirituality and mental stress in PLWH in China, who may be subject to anti-religious pressures from the government. In this study, we aimed to understand whether spirituality influences Chinese PLWH's mental stress and, if so, at what level. We recruited 200 PLWHs from Beijing's Ditan Hospital to complete a cross-sectional survey inquiring about their practice of spirituality as well as their level of mental stress. The study found that PLWH who presented with a mid-level of spirituality have the highest mental stress when compared to those who have a low level of spiritual beliefs or a high level of spiritual beliefs. This study points to the utility of healthcare providers taking PLWH's potential spirituality into consideration, perhaps in particular for those with a moderate level of spirituality, in order to provide the most comprehensive care possible

Precision engineering of precious metal catalysts for enhanced hydrogen production efficiency

This review focuses on elucidating the strategies employed for constructing efficient and stable active sites on noble metal catalysts in water electrolysis. Through the utilization of techniques such as phase modulation, morphology modulation, alloying effect, and single-atom catalysis, the electronic structure of noble metal active center atoms can be effectively tailored. This modification mitigates the strong adsorption between the active centers and reaction intermediates, thereby bolstering the intrinsic catalytic activity of the catalysts. Furthermore, the structural stability of the catalysts is enhanced through solid electronic interactions between noble metal atoms and heteroatoms. Consequently, noble metal catalysts exhibit remarkable stability during hydrogen production from water splitting in both alkaline and acidic electrolytes.ISSN:1744-3598ISSN:0957-582

Icariin induces synoviolin expression through NFE2L1 to protect neurons from ER stress-induced apoptosis.

By suppressing neuronal apoptosis, Icariin is a potential therapeutic drug for neuronal degenerative diseases. The molecular mechanisms of Icariin anti-apoptotic functions are still largely unclear. In this report, we found that Icariin induces the expression of Synoviolin, an endoplasmic reticulum (ER)-anchoring E3 ubiquitin ligase that functions as a suppressor of ER stress-induced apoptosis. The nuclear factor erythroid 2-related factor 1 (NFE2L1) is responsible for Icariin-mediated Synoviolin gene expression. Mutation of the NFE2L1-binding sites in a distal region of the Synoviolin promoter abolished Icariin-induced Synoviolin promoter activity, and knockdown of NFE2L1 expression prevented Icariin-stimulated Synoviolin expression. More importantly, Icariin protected ER stress-induced apoptosis of PC12 cells in a Synoviolin-dependent manner. Therefore, our study reveals Icariin-induced Synoviolin expression through NFE2L1 as a previously unappreciated molecular mechanism underlying the neuronal protective function of Icariin