Federated NLP in Few-shot Scenarios

Natural language processing (NLP) sees rich mobile applications. To support various language understanding tasks, a foundation NLP model is often fine-tuned in a federated, privacy-preserving setting (FL). This process currently relies on at least hundreds of thousands of labeled training samples from mobile clients; yet mobile users often lack willingness or knowledge to label their data. Such an inadequacy of data labels is known as a few-shot scenario; it becomes the key blocker for mobile NLP applications. For the first time, this work investigates federated NLP in the few-shot scenario (FedFSL). By retrofitting algorithmic advances of pseudo labeling and prompt learning, we first establish a training pipeline that delivers competitive accuracy when only 0.05% (fewer than 100) of the training data is labeled and the remaining is unlabeled. To instantiate the workflow, we further present a system FFNLP, addressing the high execution cost with novel designs. (1) Curriculum pacing, which injects pseudo labels to the training workflow at a rate commensurate to the learning progress; (2) Representational diversity, a mechanism for selecting the most learnable data, only for which pseudo labels will be generated; (3) Co-planning of a model's training depth and layer capacity. Together, these designs reduce the training delay, client energy, and network traffic by up to 46.0$\times$, 41.2$\times$ and 3000.0$\times$, respectively. Through algorithm/system co-design, FFNLP demonstrates that FL can apply to challenging settings where most training samples are unlabeled

Towards Practical Few-shot Federated NLP

Transformer-based pre-trained models have emerged as the predominant solution for natural language processing (NLP). Fine-tuning such pre-trained models for downstream tasks often requires a considerable amount of labeled private data. In practice, private data is often distributed across heterogeneous mobile devices and may be prohibited from being uploaded. Moreover, well-curated labeled data is often scarce, presenting an additional challenge. To address these challenges, we first introduce a data generator for federated few-shot learning tasks, which encompasses the quantity and skewness of scarce labeled data in a realistic setting. Subsequently, we propose AUG-FedPrompt, a prompt-based federated learning system that exploits abundant unlabeled data for data augmentation. Our experiments indicate that AUG-FedPrompt can perform on par with full-set fine-tuning with a limited amount of labeled data. However, such competitive performance comes at a significant system cost.Comment: EuroSys23 worksho

A Comprehensive Survey on Orbital Edge Computing: Systems, Applications, and Algorithms

The number of satellites, especially those operating in low-earth orbit (LEO), is exploding in recent years. Additionally, the use of COTS hardware into those satellites enables a new paradigm of computing: orbital edge computing (OEC). OEC entails more technically advanced steps compared to single-satellite computing. This feature allows for vast design spaces with multiple parameters, rendering several novel approaches feasible. The mobility of LEO satellites in the network and limited resources of communication, computation, and storage make it challenging to design an appropriate scheduling algorithm for specific tasks in comparison to traditional ground-based edge computing. This article comprehensively surveys the significant areas of focus in orbital edge computing, which include protocol optimization, mobility management, and resource allocation. This article provides the first comprehensive survey of OEC. Previous survey papers have only concentrated on ground-based edge computing or the integration of space and ground technologies. This article presents a review of recent research from 2000 to 2023 on orbital edge computing that covers network design, computation offloading, resource allocation, performance analysis, and optimization. Moreover, having discussed several related works, both technological challenges and future directions are highlighted in the field.Comment: 18 pages, 9 figures and 5 table

DialogRE^C+: An Extension of DialogRE to Investigate How Much Coreference Helps Relation Extraction in Dialogs

Dialogue relation extraction (DRE) that identifies the relations between argument pairs in dialogue text, suffers much from the frequent occurrence of personal pronouns, or entity and speaker coreference. This work introduces a new benchmark dataset DialogRE^C+, introducing coreference resolution into the DRE scenario. With the aid of high-quality coreference knowledge, the reasoning of argument relations is expected to be enhanced. In DialogRE^C+ dataset, we manually annotate total 5,068 coreference chains over 36,369 argument mentions based on the existing DialogRE data, where four different coreference chain types namely speaker chain, person chain, location chain and organization chain are explicitly marked. We further develop 4 coreference-enhanced graph-based DRE models, which learn effective coreference representations for improving the DRE task. We also train a coreference resolution model based on our annotations and evaluate the effect of automatically extracted coreference chains demonstrating the practicality of our dataset and its potential to other domains and tasks.Comment: Accepted by NLPCC 202

Study on the composition optimization method for improving the fluidity of cast Ti2_2AlNb alloy and its mechanism

In this paper, the effects of Al, Nb main elements, Fe, Mo, W, Co, B, Si and their contents on the fluidity of Ti-22Al-25Nb alloy were investigated. The composition that was beneficial to improve the fluidity was screened through the thermodynamic software calculating thermophysical parameters affecting the fluidity of Ti$_2$AlNb alloy, the numerical simulation test of its fluidity and the verification test of the fluidity of optimized alloys. Finally, the improvement mechanism of the alloy fluidity was discussed. Results showed that the appropriate reduction of Nb element was better than Al element for the improvement of fluidity. The addition of trace Fe, B and Si elements were beneficial to the improvement of fluidity, the improvement effect of B element was best, while the addition of trace Mo, W, Co were not conducive to the improvement of fluidity. The cessation mechanism of Ti$_2$AlNb alloy is the cessation mechanism of the alloy with a wide crystallization temperature range. The composition which was most beneficial to improve the fluidity was Ti-22Al-24Nb-0.1B. The main reasons for the improvement of the fluidity had two sides: on the one hand, the reduction of 1at% Nb and the addition of 0.1at% B not only increased the superheat and crystallization latent heat of the alloy, but also reduced the melt viscosity and thermal conductivity, thus improving the fluidity. On the other hand, the TiB phase refined the grains, the fine grains prevented the dendrite from growing into developed dendrite networks, inhibited the adverse effect of the increase in the width of the solidification zone on the fluidity, reduced the flow resistance of the molten metal, and further improved the fluidity of the alloy.Comment: 23 pages, 14 figures, research pape

Expression of a LINE-1 endonuclease variant in gastric cancer: its association with clinicopathological parameters

BACKGROUND: Long interspersed nuclear element-1 (LINE-1 or L1), the most abundant and only autonomously active family of non-LTR retrotransposons in the human genome, expressed not only in the germ lines but also in somatic tissues. It contributes to genetic instability, aging, and age-related diseases, such as cancer. Our previous study identified in human gastric adenocarcinoma an upregulated transcript GCRG213, which shared 88% homology with human L1 sequence and contained a putative conserved apurinic/apyrimidinic endonucleas1 domain. METHODS: Immunohistochemistry was carried out by using a monoclonal mouse anti-human GCRG213 protein (GCRG213p) antibody produced in our laboratory, on tissue microarray constructed with specimens from 175 gastric adenocarcinoma patients. The correlation between GCRG213p expression and patient clinicopathological parameters was evaluated. GCRG213p expression in gastric cancer cell lines were studied using Western blotting analysis. L1 promoter methylation status of gastric cancer cells was tested using methylation-specific PCR. BLASTP was used at the NCBI Blast server to identify GCRG213p sequence to any alignments in the Protein Data Bank databases. RESULTS: Most primary gastric cancer, lymph node metastases and gastric intestinal metaplasia glands showed positive GCRG213p immunoreactivity. High GCRG213p immunostaining score in the primary gastric cancer was positively correlated with tumor differentiation (well differentiated, p = 0.001), Lauren’s classification (intestinal type, p < 0.05) and a late age onset of gastric adenocarcinoma (≥65 yrs; p < 0.05). GCRG213p expression has no association with other clinicopathological parameters, including survival. Western blotting analysis of GCRG213p expression in gastric cancer cells indicated that GCRG213p level was higher in gastric cancer cell lines than in human normal gastric epithelium immortalized cell line GES-1. Partial methylation of L1 in gastric cancer cells was confirmed by methylation-specific PCR. BLASTP program analysis revealed that GCRG213p peptide shared 83.0% alignment with the C-terminal region of L1 endonuclease (L1-EN). GCRG213p sequence possesses the important residues that compose the conserved features of L1-EN. CONCLUSIONS: GCRG213p could be a variant of L1-EN, a functional member of L1-EN family. Overexpression of GCRG213p is common in both primary gastric cancer and lymph node metastasis. These findings provide evidence of somatic L1 expression in gastric cancer, and its potential consequences in the form of tumor

Beneficial effects of baicalein on a model of allergic rhinitis

Allergic rhinitis (AR) is a common disease that causes severe inflammation and even disabilities. Previous studies have reported baicalein to have an anti-inflammatory effect. However, the pharmacological action of baicalein on anaphylaxis has not been clarified yet. This study assessed the in vivo protective effect of baicalein post-treatment in an ameliorating ovalbumin (OVA)-sensitized AR rat model. Baicalein attenuated histological alterations, aberrant tissue repair and inflammation after OVA-induced AR. Baicalein reduced the frequency of nasal/ear rubs and sneezes in rats, and inhibited generation of several inflammatory cytokines (TNF-α, IL-1β, and IL-6) in both blood and nasal lavage of rats. Infiltrations of eosinophils, lymphocyte, and neutrophils were decreased in baicalein-administered rats. Furthermore, baicalein inhibited the expression of STAT3 phosphorylation in the nasal mucosa. In summary, baicalein attenuated OVA-induced AR and inflammation, which suggests it as a promising therapeutic agent for the alleviation of AR-associated inflammation and pathology