A Quantitative Review on Language Model Efficiency Research

Language models (LMs) are being scaled and becoming powerful. Improving their efficiency is one of the core research topics in neural information processing systems. Tay et al. (2022) provided a comprehensive overview of efficient Transformers that have become an indispensable staple in the field of NLP. However, in the section of "On Evaluation", they left an open question "which fundamental efficient Transformer one should consider," answered by "still a mystery" because "many research papers select their own benchmarks." Unfortunately, there was not quantitative analysis about the performances of Transformers on any benchmarks. Moreover, state space models (SSMs) have demonstrated their abilities of modeling long-range sequences with non-attention mechanisms, which were not discussed in the prior review. This article makes a meta analysis on the results from a set of papers on efficient Transformers as well as those on SSMs. It provides a quantitative review on LM efficiency research and gives suggestions for future research.Comment: 29 pages, 24 table

Data Augmentation for Graph Neural Networks

Data augmentation has been widely used to improve generalizability of machine learning models. However, comparatively little work studies data augmentation for graphs. This is largely due to the complex, non-Euclidean structure of graphs, which limits possible manipulation operations. Augmentation operations commonly used in vision and language have no analogs for graphs. Our work studies graph data augmentation for graph neural networks (GNNs) in the context of improving semi-supervised node-classification. We discuss practical and theoretical motivations, considerations and strategies for graph data augmentation. Our work shows that neural edge predictors can effectively encode class-homophilic structure to promote intra-class edges and demote inter-class edges in given graph structure, and our main contribution introduces the GAug graph data augmentation framework, which leverages these insights to improve performance in GNN-based node classification via edge prediction. Extensive experiments on multiple benchmarks show that augmentation via GAug improves performance across GNN architectures and datasets.Comment: AAAI 2021. This complete version contains the Appendi

Research on the characteristics and application of food materials in food 3D printing

Food material is the key factor of food 3D printing. This review summarized the characteristics of food materials (protein, starch, hydrogel and fat) commonly used in food 3D printing at present, analyzed the influence of different material combinations on the forming performance. Moreover, suggestions on how to improve the printing effect of difficult printing materials were also provided, and the technical bottleneck of food materials development faced in food 3D printing nowadays were noticed. Finally, the direction of food 3D printing technology in the future was prospected

Molecular Targeted Therapy in the Treatment of Chordoma: A Systematic Review

Objectives: Chordoma is a rare bone malignancy that affects the spine and skull base. Treatment dilemma leads to a high rate of local relapse and distant metastases. Molecular targeted therapy (MTT) is an option for advanced chordoma, but its therapeutic efficacy and safety have not been investigated systematically. Therefore, a systematic review was conducted on studies reporting MTT regimens for chordoma.Methods: Clinical trials, case series and case reports on chordoma MTT were identified using MEDLINE, Cochrane library and EMBASE, and systematically reviewed. Data on clinical outcomes, such as median overall survival, progression-free survival, response rate and adverse events (AEs) were extracted and analyzed.Results: Thirty-three eligible studies were selected for the systematic review, which indicated that imatinib and erlotinib were the most frequently used molecular targeted inhibitors (MTIs) for chordoma. For PDGFR-positive and/or EGFR-positive chordoma, clinical benefits were achieved with acceptable AEs. Monotherapy is preferred as the first-line of treatment, and combined drug therapy as the second-line treatment. In addition, the brachyury vaccine has shown promising results.Conclusions: The selection of MTIs for patients with advanced or relapsed chordoma should be based on gene mutation screening and immunohistochemistry (IHC). Monotherapy of TKIs is recommended as the first-line management, and combination therapy (two TKIs or TKI plus mTOR inhibitor) may be the choice for drug-resistant chordoma. Brachyury vaccine is a promising therapeutic strategy and requires more clinical trials to evaluate its safety and efficacy

SCNet: Sparse Compression Network for Music Source Separation

Deep learning-based methods have made significant achievements in music source separation. However, obtaining good results while maintaining a low model complexity remains challenging in super wide-band music source separation. Previous works either overlook the differences in subbands or inadequately address the problem of information loss when generating subband features. In this paper, we propose SCNet, a novel frequency-domain network to explicitly split the spectrogram of the mixture into several subbands and introduce a sparsity-based encoder to model different frequency bands. We use a higher compression ratio on subbands with less information to improve the information density and focus on modeling subbands with more information. In this way, the separation performance can be significantly improved using lower computational consumption. Experiment results show that the proposed model achieves a signal to distortion ratio (SDR) of 9.0 dB on the MUSDB18-HQ dataset without using extra data, which outperforms state-of-the-art methods. Specifically, SCNet's CPU inference time is only 48% of HT Demucs, one of the previous state-of-the-art models.Comment: Accepted by ICASSP 202

ZNF750 is a lineage-specific tumour suppressor in squamous cell carcinoma.

ZNF750 controls epithelial homeostasis by regulating epidermal-differentiation genes, a role underscored by its pathogenic mutations in esophageal squamous cell cancers (SCCs). However, the precise role of ZNF750 in SCC cell biology remains unclear. In this study, we report that ZNF750 is exclusively deleted, mutated and underexpressed in human SCCs, and low ZNF750 expression is associated with poor survival. Restoration of wildtype, but not mutant ZNF750 protein uniquely inhibited the malignant phenotypes of SCC cells both in vitro and in vivo. Notably, ZNF750 promoted the expression of a long non-coding RNA (TINCR), which mediated both cancer-inhibition and differentiation-induction effects of ZNF750. In addition, ZNF750 potently suppressed cell migration by directly inhibiting the transactivation of LAMC2. Together, our findings characterize ZNF750 as a crucial SCC-specific suppressor and uncover its novel anticancer-associated functions