Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model

Pre-trained multilingual language models play an important role in cross-lingual natural language understanding tasks. However, existing methods did not focus on learning the semantic structure of representation, and thus could not optimize their performance. In this paper, we propose Multi-level Multilingual Knowledge Distillation (MMKD), a novel method for improving multilingual language models. Specifically, we employ a teacher-student framework to adopt rich semantic representation knowledge in English BERT. We propose token-, word-, sentence-, and structure-level alignment objectives to encourage multiple levels of consistency between source-target pairs and correlation similarity between teacher and student models. We conduct experiments on cross-lingual evaluation benchmarks including XNLI, PAWS-X, and XQuAD. Experimental results show that MMKD outperforms other baseline models of similar size on XNLI and XQuAD and obtains comparable performance on PAWS-X. Especially, MMKD obtains significant performance gains on low-resource languages.Comment: accepted at EMNLP 202

RLLTE: Long-Term Evolution Project of Reinforcement Learning

We present RLLTE: a long-term evolution, extremely modular, and open-source framework for reinforcement learning (RL) research and application. Beyond delivering top-notch algorithm implementations, RLLTE also serves as a toolkit for developing algorithms. More specifically, RLLTE decouples the RL algorithms completely from the exploitation-exploration perspective, providing a large number of components to accelerate algorithm development and evolution. In particular, RLLTE is the first RL framework to build a complete and luxuriant ecosystem, which includes model training, evaluation, deployment, benchmark hub, and large language model (LLM)-empowered copilot. RLLTE is expected to set standards for RL engineering practice and be highly stimulative for industry and academia.Comment: 22 pages, 15 figure

Serum protein biomarkers for HCC risk prediction in HIV/HBV co-infected people: a clinical proteomic study using mass spectrometry

BackgroundHBV coinfection is frequent in people living with HIV (PLWH) and is the leading cause of hepatocellular carcinoma (HCC). While risk prediction methods for HCC in patients with HBV monoinfection have been proposed, suitable biomarkers for early diagnosis of HCC in PLWH remain uncommon.MethodsLiquid chromatography-tandem mass spectrometry (LC-MS/MS) was used to examine serum protein alterations in HCC and non-HCC patients with HIV and HBV co-infection. Gene Ontology (GO), Kyoto Encyclopedia of Genes and Genomes (KEGG), and Disease Ontology (DO) enrichment analysis were performed on the differentially expressed proteins (DEPs). The risk prediction model was created using five-cross-validation and LASSO regression to filter core DEPs.ResultsA total of 124 DEPs were discovered, with 95 proteins up-regulated and 29 proteins down-regulated. Extracellular matrix organization and membrane component were the DEPs that were most abundant in the categories of biological processes (BP) and cellular components (CC). Proteoglycans in cancer were one of the top three DEPs primarily enriched in the KEGG pathway, and 60.0% of DEPs were linked to various neoplasms in terms of DO enrichment. Eleven proteins, including GAPR1, PLTP, CLASP2, IGHV1-69D, IGLV5-45, A2M, VNN1, KLK11, ANPEP, DPP4 and HYI, were chosen as the core DEPs, and a nomogram was created to predict HCC risk.ConclusionIn HIV/HBV patients with HCC, several differential proteins can be detected in plasma by mass spectrometry, which can be used as screening markers for early diagnosis and risk prediction of HCC. Monitoring protease expression differences can help in the diagnosis and prognosis of HCC

Neutrophil-to-lymphocyte ratio as an independent risk factor for mortality in hospitalized patients with COVID-19

Background: Several studies have described the clinical characteristics of patients with novel coronavirus (SARS-CoV-2) infected pneumonia (COVID-19), indicating severe patients tended to have higher neutrophil to lymphocyte ratio (NLR). Whether baseline NLR could be an independent predictor of in-hospital death in Chinese COVID-19 patients remains to be investigated. Methods: A cohort of patients with COVID-19 admitted to the Zhongnan Hospital of Wuhan University from January 1 to February 29 was retrospectively analyzed. The baseline data of laboratory examinations, including NLR, were collected. Univariate and multivariate logistic regression models were developed to assess the independent relationship between the baseline NLR and in-hospital all-cause death. A sensitivity analysis was performed by converting NLR from a continuous variable to a categorical variable according to tertile. Interaction and stratified analyses were conducted as well. Results: 245 COVID-19 patients were included in the final analyses, and the in-hospital mortality was 13.47%. Multivariate analysis demonstrated that there was 8% higher risk of in-hospital mortality for each unit increase in NLR (Odds ratio [OR] = 1.08; 95% confidence interval [95% CI], 1.01 to 1.14; P = 0.0147). Compared with patients in the lowest tertile, the NLR of patients in the highest tertile had a 15.04-fold higher risk of death (OR = 16.04; 95% CI, 1.14 to 224.95; P = 0.0395) after adjustment for potential confounders. Notably, the fully adjusted OR for mortality was 1.10 in males for each unit increase of NLR (OR = 1.10; 95% CI, 1.02 to 1.19; P = 0.016). Conclusions: NLR is an independent risk factor of the in-hospital mortality for COVID-19 patients especially for male. Assessment of NLR may help identify high risk individuals with COVID-19

Multi-part and scale adaptive visual tracker based on kernel correlation filter.

Accurate visual tracking is a challenging issue in computer vision. Correlation filter (CF) based methods are sought in visual tracking based on their efficiency and high performance. Nonetheless, CF-based trackers are sensitive to partial occlusion, which may reduce their overall performance and even lead to failure in tracking challenge. In this paper, we presented a very powerful tracker based on the kernelized correlation filter tracker (KCF). Firstly, we employ an intelligent multi-part tracking algorithm to improve the overall capability of correlation filter based tracker, especially in partial-occlusion challenges. Secondly, to cope with the problem of scale variation, we employ an effective scale adaptive scheme, which divided the target into four patches and computed the scale factor by finding the maximum response position of each patch via kernelized correlation filter. With this method, the scale computation was transformed into locating the centers of the patches. Thirdly, because the small deviation of the central function value will bring the problem of location ambiguity. To solve this problem, the new Gaussian kernel functions are introduced in this paper. Experiments on the default 51 video sequences in Visual Tracker Benchmark demonstrate that our proposed tracker provides significant improvement compared with the state-of-art trackers

Adaptive Multi-part Target Representation for Tracking

In this paper we propose and evaluate an effective approach based on multiple colour histograms. The target is adaptively divided into non-overlapping regions. The proposed partition does not weaken the robustness of the colour histogram representation; it can be used to any class targets. Experimental results show that the proposed representation improve the tracking accuracy and decrease the number of iterations

Adaptive Scale Compressive Tracking with Feature Integration

Compressive tracking (CT) is utilized to cope with real-time tracking, which use a very sparse measurement matrix to compressive samples of targets and background, then a classifier is trained to distinguish foreground and background. However, this algorithm suffers from the drifting problem, and used the fixed size tracking box to detect, recognize, and update the samples and classifier. In order to solve these problems, we adopt a different way to extracted positive samples, and employ powerful features to exploit the advantages of feature fusion to describe target, a scale pyramid is used to realize adaptive scale tracking. Experimental results on various benchmark video sequences demonstrate the superior performance of our algorithm

Adaptive Multi-part Target Representation for Tracking

In this paper we propose and evaluate an effective approach based on multiple colour histograms. The target is adaptively divided into non-overlapping regions. The proposed partition does not weaken the robustness of the colour histogram representation; it can be used to any class targets. Experimental results show that the proposed representation improve the tracking accuracy and decrease the number of iterations

Adaptive Scale Compressive Tracking with Feature Integration

Compressive tracking (CT) is utilized to cope with real-time tracking, which use a very sparse measurement matrix to compressive samples of targets and background, then a classifier is trained to distinguish foreground and background. However, this algorithm suffers from the drifting problem, and used the fixed size tracking box to detect, recognize, and update the samples and classifier. In order to solve these problems, we adopt a different way to extracted positive samples, and employ powerful features to exploit the advantages of feature fusion to describe target, a scale pyramid is used to realize adaptive scale tracking. Experimental results on various benchmark video sequences demonstrate the superior performance of our algorithm