Continual Learning with Strong Experience Replay

Continual Learning (CL) aims at incrementally learning new tasks without forgetting the knowledge acquired from old ones. Experience Replay (ER) is a simple and effective rehearsal-based strategy, which optimizes the model with current training data and a subset of old samples stored in a memory buffer. To further reduce forgetting, recent approaches extend ER with various techniques, such as model regularization and memory sampling. However, the prediction consistency between the new model and the old one on current training data has been seldom explored, resulting in less knowledge preserved when few previous samples are available. To address this issue, we propose a CL method with Strong Experience Replay (SER), which additionally utilizes future experiences mimicked on the current training data, besides distilling past experience from the memory buffer. In our method, the updated model will produce approximate outputs as its original ones, which can effectively preserve the acquired knowledge. Experimental results on multiple image classification datasets show that our SER method surpasses the state-of-the-art methods by a noticeable margin

Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation

For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e.g. mBART, the self-supervised pretraining task is trained on a wide range of monolingual languages, e.g. 25 languages from commoncrawl, while the downstream cross-lingual tasks generally progress on a bilingual language subset, e.g. English-German, making there exists the cross-lingual data discrepancy, namely \textit{domain discrepancy}, and cross-lingual learning objective discrepancy, namely \textit{task discrepancy}, between the pretrain and finetune stages. To bridge the above cross-lingual domain and task gaps, we extend the vanilla pretrain-finetune pipeline with extra code-switching restore task. Specifically, the first stage employs the self-supervised code-switching restore task as a pretext task, allowing the multilingual Seq2Seq PLM to acquire some in-domain alignment information. And for the second stage, we continuously fine-tune the model on labeled data normally. Experiments on a variety of cross-lingual NLG tasks, including 12 bilingual translation tasks, 36 zero-shot translation tasks, and cross-lingual summarization tasks show our model outperforms the strong baseline mBART consistently. Comprehensive analyses indicate our approach could narrow the cross-lingual sentence representation distance and improve low-frequency word translation with trivial computational cost

Optimization of nano coating to reduce the thermal deformation of ball screws

To reduce the thermal deformation of ball screws, the process of nano coating preparation for coating on ball screws to reduce temperature rise and thereby thermal deformation was discussed in this article. Simultaneously, the cooling mechanism was presented. The thermal channels and the relatively even distribution of graphene in the nano coating were observed in scanning electron microscopic images. In terms of the preparation of nano coating, optimization design was carried out to obtain the optimized material ratio and nozzle flow through orthogonal experiment. The influence of design parameters of nano coating on reducing thermal deformation was also discussed. The experimental results show that the maximum temperature rise, thermal deformation, and time to reach thermal balance decreased by 12.5, 69.1, and 46.3%, respectively. The effectiveness of nano coating in reducing thermal deformation was validated experimentally

Pulmonary alveolar type I cell population consists of two distinct subtypes that differ in cell fate.

Pulmonary alveolar type I (AT1) cells cover more than 95% of alveolar surface and are essential for the air-blood barrier function of lungs. AT1 cells have been shown to retain developmental plasticity during alveolar regeneration. However, the development and heterogeneity of AT1 cells remain largely unknown. Here, we conducted a single-cell RNA-seq analysis to characterize postnatal AT1 cell development and identified insulin-like growth factor-binding protein 2 (Igfbp2) as a genetic marker specifically expressed in postnatal AT1 cells. The portion of AT1 cells expressing Igfbp2 increases during alveologenesis and in post pneumonectomy (PNX) newly formed alveoli. We found that the adult AT1 cell population contains both Hopx+Igfbp2+ and Hopx+Igfbp2- AT1 cells, which have distinct cell fates during alveolar regeneration. Using an Igfbp2-CreER mouse model, we demonstrate that Hopx+Igfbp2+ AT1 cells represent terminally differentiated AT1 cells that are not able to transdifferentiate into AT2 cells during post-PNX alveolar regeneration. Our study provides tools and insights that will guide future investigations into the molecular and cellular mechanism or mechanisms underlying AT1 cell fate during lung development and regeneration

Unlikelihood Tuning on Negative Samples Amazingly Improves Zero-Shot Translation

Zero-shot translation (ZST), which is generally based on a multilingual neural machine translation model, aims to translate between unseen language pairs in training data. The common practice to guide the zero-shot language mapping during inference is to deliberately insert the source and target language IDs, e.g., for English and for German. Recent studies have shown that language IDs sometimes fail to navigate the ZST task, making them suffer from the off-target problem (non-target language words exist in the generated translation) and, therefore, difficult to apply the current multilingual translation model to a broad range of zero-shot language scenarios. To understand when and why the navigation capabilities of language IDs are weakened, we compare two extreme decoder input cases in the ZST directions: Off-Target (OFF) and On-Target (ON) cases. By contrastively visualizing the contextual word representations (CWRs) of these cases with teacher forcing, we show that 1) the CWRs of different languages are effectively distributed in separate regions when the sentence and ID are matched (ON setting), and 2) if the sentence and ID are unmatched (OFF setting), the CWRs of different languages are chaotically distributed. Our analyses suggest that although they work well in ideal ON settings, language IDs become fragile and lose their navigation ability when faced with off-target tokens, which commonly exist during inference but are rare in training scenarios. In response, we employ unlikelihood tuning on the negative (OFF) samples to minimize their probability such that the language IDs can discriminate between the on- and off-target tokens during training. Experiments spanning 40 ZST directions show that our method reduces the off-target ratio by -48.0% on average, leading to a +9.1 BLEU improvement with only an extra +0.3% tuning cost

Preoperative Imaging for Thoracic Branch of Supraclavicular Artery Flap A Comparative Study of Contrast-Enhanced Ultrasound With Three-Dimensional Reconstruction and Color Duplex Ultrasound

Objective: The thoracic branch of supraclavicular artery (TBSA) flap has been widely used to reconstruct face and neck defects. However, the branches of the supraclavicular artery (SCA) exhibit considerable anatomical variations. The aim of this study was to evaluate and compare the role of contrast-enhanced ultrasound (CEUS) with 3-dimensional (3D) reconstruction and regular color duplex ultrasonography (CDUS) in the preoperative assessment of TBSA flap. Methods: From May 2009 to October 2013, 20 patients (involving 26 flaps) receiving anterior chest flaps for lower face and neck reconstruction underwent both CDUS and CEUS with 3D reconstruction preoperatively for detecting the TBSAs. The number of TBSAs, their caliber, peak systolic velocity (PSV), and course were recorded. In case of an absent TBSA, the second and third perforators of the internal mammary artery were detected. The preoperative imaging data were compared with the intraoperative findings to evaluate the value of CDUS and CEUS with 3D reconstruction for planning and performing the TBSA flaps. All patients were followed up for more than 1 year. Results: A total of 37 TBSAs in 16 flaps were found by CDUS with a mean caliber of 0.6 +/- 0.1 mm and a mean PSV of 13.1 +/- 1.6 cm/s, whereas 48 TBSAs in 20 flaps were found by CEUS with a mean caliber of 0.8 +/- 0.2 mm and a mean PSVof 12.5 +/- 2.1 cm/sec. In 18 flaps with TBSA PSV above 10 cm/s, pedicled TBSA flaps were performed, whereas pedicled or free internal mammary artery flaps were chosen as alternative for the remaining 8 flaps. All 48 TBSAs were found intraoperatively and their origin from the SCA confirmed, indicating specificity and positive predictive value of both CDUS and CEUS were 100% in localizing TBSA preoperatively, whereas sensitivity and negative predictive value of CEUS were higher than using CDUS. Conclusions: The branches of SCA have marked anatomical variations. CEUS with 3D reconstruction has advantages over CDUS for the preoperative assessment of the donor-site vascular supply of TBSA flaps

Extracellular RNA in melanoma: Advances, challenges, and opportunities

Melanoma, a malignant mass lesion that originates in melanocytes and has a high rate of malignancy, metastasis, and mortality, is defined by these characteristics. Malignant melanoma is a kind of highly malignant tumor that produces melanin and has a high mortality rate. Its incidence accounts for 1%–3% of all malignant tumors and shows an obvious upward trend. The discovery of biomolecules for the diagnosis and treatment of malignant melanoma has important application value. So far, the exact molecular mechanism of melanoma development relevant signal pathway still remains unclear. According to previous studies, extracellular RNAs (exRNAs) have been implicated in tumorigenesis and spread of melanoma. They can influence the proliferation, invasion and metastasis of melanoma by controlling the expression of target genes and can also influence tumor progression by participating in signal transduction mechanisms. Therefore, understanding the relationship between exRNA and malignant melanoma and targeting therapy is of positive significance for its prevention and treatment. In this review, we did an analysis of extracellular vesicles of melanoma which focused on the role of exRNAs (lncRNAs, miRNAs, and mRNAs) and identifies several potential therapeutic targets. In addition, we discuss the typical signaling pathways involved in exRNAs, advances in exRNA detection and how they affect the tumor immune microenvironment in melanoma