A-SFS: Semi-supervised Feature Selection based on Multi-task Self-supervision

Feature selection is an important process in machine learning. It builds an interpretable and robust model by selecting the features that contribute the most to the prediction target. However, most mature feature selection algorithms, including supervised and semi-supervised, fail to fully exploit the complex potential structure between features. We believe that these structures are very important for the feature selection process, especially when labels are lacking and data is noisy. To this end, we innovatively introduce a deep learning-based self-supervised mechanism into feature selection problems, namely batch-Attention-based Self-supervision Feature Selection(A-SFS). Firstly, a multi-task self-supervised autoencoder is designed to uncover the hidden structure among features with the support of two pretext tasks. Guided by the integrated information from the multi-self-supervised learning model, a batch-attention mechanism is designed to generate feature weights according to batch-based feature selection patterns to alleviate the impacts introduced by a handful of noisy data. This method is compared to 14 major strong benchmarks, including LightGBM and XGBoost. Experimental results show that A-SFS achieves the highest accuracy in most datasets. Furthermore, this design significantly reduces the reliance on labels, with only 1/10 labeled data needed to achieve the same performance as those state of art baselines. Results show that A-SFS is also most robust to the noisy and missing data.Comment: 18 pages, 7 figures, accepted by knowledge-based system

A Survey on Causal Reinforcement Learning

While Reinforcement Learning (RL) achieves tremendous success in sequential decision-making problems of many domains, it still faces key challenges of data inefficiency and the lack of interpretability. Interestingly, many researchers have leveraged insights from the causality literature recently, bringing forth flourishing works to unify the merits of causality and address well the challenges from RL. As such, it is of great necessity and significance to collate these Causal Reinforcement Learning (CRL) works, offer a review of CRL methods, and investigate the potential functionality from causality toward RL. In particular, we divide existing CRL approaches into two categories according to whether their causality-based information is given in advance or not. We further analyze each category in terms of the formalization of different models, ranging from the Markov Decision Process (MDP), Partially Observed Markov Decision Process (POMDP), Multi-Arm Bandits (MAB), and Dynamic Treatment Regime (DTR). Moreover, we summarize the evaluation matrices and open sources while we discuss emerging applications, along with promising prospects for the future development of CRL.Comment: 29 pages, 20 figure

Effect of dexmedetomidine on miR-144-3p expression and epithelial mesenchymal transition in gastric cancer cells

Purpose: To investigate the effect of dexmedetomidine (DEX) on epithelial mesenchymal transition (EMT) in gastric cancer cells, and the role of microRNA-144-3p (miR-144-3p) in the process.Methods: The effect of DEX on miRNA expression profile was analyzed using GEO database(https://www.ncbi.nlm.nih.gov/gds/). Human gastric cancer cells were cultured in vitro, and one group of cells was treated with saline for 48 h (control group). Cells treated with DEX at doses of 0.01, 0.1 and 1.0 μmol/L for 48 h were marked as low-, medium- and high-DEX concentration groups. The mRNA expression levels of miR-144-3p, ZEB1, E-cadherin and vimentin were determined using real-time quantitative polymerase chain reaction (RT-PCR), while the protein expressions of ZEB1, E-cadherin and vimentin were assayed with Western blotting. Cell proliferation was determined with CCK-8 assay, while metastasis was measured using Transwell assay.Results: The GEO database demonstrated that the expression of miR-144-3p in rat cardiomyocytes was significantly decreased after DEX treatment (p < 0.05). The expression of miR-144-3p was decreased in all groups, when compared to the control group, but the expressions of ZEB1 and vimentin were increased, while that of E-cadherin was down-regulated (p < 0.05). Cell proliferation in the high-DEX concentration group was decreased (p < 0.05). The degrees of cell invasion and migration were increased in the medium- and high-DEX concentration groups (p < 0.05).Conclusion: DEX promotes the metastasis of gastric cancer cells by regulation of epithelialmesenchymal transition (EMT) and the expression of miR-144-3p. This finding provides a new insight into the treatment of gastric cancer

Learning to Skip for Language Modeling

Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning. However, most language models allocate the same amount of parameters or computation to each token, disregarding the complexity or importance of the input data. We argue that in language model pretraining, a variable amount of computation should be assigned to different tokens, and this can be efficiently achieved via a simple routing mechanism. Different from conventional early stopping techniques where tokens can early exit at only early layers, we propose a more general method that dynamically skips the execution of a layer (or module) for any input token with a binary router. In our extensive evaluation across 24 NLP tasks, we demonstrate that the proposed method can significantly improve the 1-shot performance compared to other competitive baselines only at mild extra cost for inference

Effective inhibition of HCMV UL49 gene expression and viral replication by oligonucleotide external guide sequences and RNase P

Abstract Background Human cytomegalovirus (HCMV) is a ubiquitous herpesvirus that typically causes asymptomatic infections in healthy individuals but may lead to serious complications in newborns and immunodeficient individuals. The emergence of drug-resistant strains of HCMV has posed a need for the development of new drugs and treatment strategies. Antisense molecules are promising gene-targeting agents for specific regulation of gene expression. External guide sequences (EGSs) are oligonucleotides that consist of a sequence complementary to a target mRNA and recruit intracellular RNase P for specific degradation of the target RNA. The UL49-deletion BAC of HCMV was significantly defective in growth in human foreskin fibroblasts. Therefore, UL49 gene may serve as a potential target for novel drug development to combat HCMV infection. In this study, DNA-based EGS molecules were synthesized to target the UL49 mRNA of human cytomegalovirus (HCMV). Results By cleavage activity assessing in vitro, the EGS aimed to the cleavage site 324 nt downstream from the translational initiation codon of UL49 mRNA (i.e. EGS324) was confirmed be efficient to direct human RNase P to cleave the target mRNA sequence. When EGS324 was exogenously administered into HCMV-infected human foreskin fibroblasts (HFFs), a significant reduction of ~76% in the mRNA and ~80% in the protein expression of UL49 gene, comparing with the cells transfected with control EGSs. Furthermore, a reduction of about 330-fold in HCMV growth were observed in HCMV-infected HFFs treated with the EGS. Conclusions These results indicated that UL49 gene was essential for replication of HCMV. Moreover, our study provides evidence that exogenous administration of a DNA-based EGS can be used as a potential therapeutic approach for inhibiting gene expression and replication of a human virus.</p

Experimental Study of Granular Clogging in Two-Dimensional Hopper

We experimentally investigate the clogging process of granular materials in a two-dimensional hopper, and present a self-consistent physical mechanism of clogging based on preformed dynamic chain structures in the flow. We found that these chain structures follow a specific modified restricted random walk, and clogging occurs when they are mechanically stable enough to withstand the flow fluctuations, resulting in the formation of an arch at the outlet. We introduce a simple model which can explain the clogging probability by incorporating an analytical expression for chain formation and its transition into an arch. Our results provide insight into the microscopic mechanism of clogging in hopper flow.Comment: 22 pages, 8 figure