SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization

Transfer learning has fundamentally changed the landscape of natural language processing (NLP) research. Many existing state-of-the-art models are first pre-trained on a large text corpus and then fine-tuned on downstream tasks. However, due to limited data resources from downstream tasks and the extremely large capacity of pre-trained models, aggressive fine-tuning often causes the adapted model to overfit the data of downstream tasks and forget the knowledge of the pre-trained model. To address the above issue in a more principled manner, we propose a new computational framework for robust and efficient fine-tuning for pre-trained language models. Specifically, our proposed framework contains two important ingredients: 1. Smoothness-inducing regularization, which effectively manages the capacity of the model; 2. Bregman proximal point optimization, which is a class of trust-region methods and can prevent knowledge forgetting. Our experiments demonstrate that our proposed method achieves the state-of-the-art performance on multiple NLP benchmarks.Comment: The 58th annual meeting of the Association for Computational Linguistics (ACL 2020

2-.mu.m fiber amplified spontaneous emission (ASE) source

A 2-.mu.m fiber Amplified Spontaneous Emission (ASE) source provides a wide emission bandwidth and improved spectral stability/purity for a given output power. The fiber ASE source is formed from a heavy metal oxide multicomponent glass selected from germanate, tellurite and bismuth oxides and doped with high concentrations, 0.5-15 wt. %, thulium oxides (Tm.sub.2O.sub.3) or 0.1-5 wt% holmium oxides (Ho.sub.2O.sub.3) or mixtures thereof. The high concentration of thulium dopants provide highly efficient pump absorption and high quantum efficiency. Co-doping of Tm and Ho can broaden the ASE spectrum

Eradication of unresectable liver metastasis through induction of tumour specific energy depletion.

Treatment of liver metastasis experiences slow progress owing to the severe side effects. In this study, we demonstrate a strategy capable of eliminating metastatic cancer cells in a selective manner. Nucleus-targeting W18O49 nanoparticles (WONPs) are conjugated to mitochondria-selective mesoporous silica nanoparticles (MSNs) containing photosensitizer (Ce6) through a Cathepsin B-cleavable peptide. In hepatocytes, upon the laser irradiation, the generated singlet oxygen species are consumed by WONPs, in turn leading to the loss of their photothermally heating capacity, thereby sparing hepatocyte from thermal damage induced by the laser illumination. By contrast, in cancer cells, the cleaved peptide linker allows WONPs and MSNs to respectively target nucleus and mitochondria, where the therapeutic powers could be unleashed, both photodynamically and photothermally. This ensures the energy production of cancer cells can be abolished. We further assess the underlying molecular mechanism at both gene and protein levels to better understand the therapeutic outcome

Dual Skipping Networks

Inspired by the recent neuroscience studies on the left-right asymmetry of the human brain in processing low and high spatial frequency information, this paper introduces a dual skipping network which carries out coarse-to-fine object categorization. Such a network has two branches to simultaneously deal with both coarse and fine-grained classification tasks. Specifically, we propose a layer-skipping mechanism that learns a gating network to predict which layers to skip in the testing stage. This layer-skipping mechanism endows the network with good flexibility and capability in practice. Evaluations are conducted on several widely used coarse-to-fine object categorization benchmarks, and promising results are achieved by our proposed network model.Comment: CVPR 2018 (poster); fix typ

Identification of novel gene signature for lung adenocarcinoma by machine learning to predict immunotherapy and prognosis

BackgroundLung adenocarcinoma (LUAD) as a frequent type of lung cancer has a 5-year overall survival rate of lower than 20% among patients with advanced lung cancer. This study aims to construct a risk model to guide immunotherapy in LUAD patients effectively.Materials and methodsLUAD Bulk RNA-seq data for the construction of a model, single-cell RNA sequencing (scRNA-seq) data (GSE203360) for cell cluster analysis, and microarray data (GSE31210) for validation were collected from The Cancer Genome Atlas (TCGA) and Gene Expression Omnibus (GEO) database. We used the Seurat R package to filter and process scRNA-seq data. Sample clustering was performed in the ConsensusClusterPlus R package. Differentially expressed genes (DEGs) between two groups were mined by the Limma R package. MCP-counter, CIBERSORT, ssGSEA, and ESTIMATE were employed to evaluate immune characteristics. Stepwise multivariate analysis, Univariate Cox analysis, and Lasso regression analysis were conducted to identify key prognostic genes and were used to construct the risk model. Key prognostic gene expressions were explored by RT-qPCR and Western blot assay.ResultsA total of 27 immune cell marker genes associated with prognosis were identified for subtyping LUAD samples into clusters C3, C2, and C1. C1 had the longest overall survival and highest immune infiltration among them, followed by C2 and C3. Oncogenic pathways such as VEGF, EFGR, and MAPK were more activated in C3 compared to the other two clusters. Based on the DEGs among clusters, we confirmed seven key prognostic genes including CPA3, S100P, PTTG1, LOXL2, MELTF, PKP2, and TMPRSS11E. Two risk groups defined by the seven-gene risk model presented distinct responses to immunotherapy and chemotherapy, immune infiltration, and prognosis. The mRNA and protein level of CPA3 was decreased, while the remaining six gene levels were increased in clinical tumor tissues.ConclusionImmune cell markers are effective in clustering LUAD samples into different subtypes, and they play important roles in regulating the immune microenvironment and cancer development. In addition, the seven-gene risk model may serve as a guide for assisting in personalized treatment in LUAD patients

Lightweight Three-Factor Authentication and Key Agreement Protocol for Internet-Integrated Wireless Sensor Networks

Wireless sensor networks (WSNs) will be integrated into the future Internet as one of the components of the Internet of Things, and will become globally addressable by any entity connected to the Internet. Despite the great potential of this integration, it also brings new threats, such as the exposure of sensor nodes to attacks originating from the Internet. In this context, lightweight authentication and key agreement protocols must be in place to enable end-to-end secure communication. Recently, Amin et al. proposed a three-factor mutual authentication protocol for WSNs. However, we identified several flaws in their protocol. We found that their protocol suffers from smart card loss attack where the user identity and password can be guessed using offline brute force techniques. Moreover, the protocol suffers from known session-specific temporary information attack, which leads to the disclosure of session keys in other sessions. Furthermore, the protocol is vulnerable to tracking attack and fails to fulfill user untraceability. To address these deficiencies, we present a lightweight and secure user authentication protocol based on the Rabin cryptosystem, which has the characteristic of computational asymmetry. We conduct a formal verification of our proposed protocol using ProVerif in order to demonstrate that our scheme fulfills the required security properties. We also present a comprehensive heuristic security analysis to show that our protocol is secure against all the possible attacks and provides the desired security features. The results we obtained show that our new protocol is a secure and lightweight solution for authentication and key agreement for Internet-integrated WSNs