Towards Personalized Federated Learning via Heterogeneous Model Reassembly

This paper focuses on addressing the practical yet challenging problem of model heterogeneity in federated learning, where clients possess models with different network structures. To track this problem, we propose a novel framework called pFedHR, which leverages heterogeneous model reassembly to achieve personalized federated learning. In particular, we approach the problem of heterogeneous model personalization as a model-matching optimization task on the server side. Moreover, pFedHR automatically and dynamically generates informative and diverse personalized candidates with minimal human intervention. Furthermore, our proposed heterogeneous model reassembly technique mitigates the adverse impact introduced by using public data with different distributions from the client data to a certain extent. Experimental results demonstrate that pFedHR outperforms baselines on three datasets under both IID and Non-IID settings. Additionally, pFedHR effectively reduces the adverse impact of using different public data and dynamically generates diverse personalized models in an automated manner

Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations

This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD robustness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pre-trained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate 5 classic methods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, we evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data is available, fine-tuning domain-specific models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in-context learning yields better results. We identify that both fine-tuned small models and LLMs face challenges in effectively addressing downstream tasks. The code is public at \url{https://github.com/lifan-yuan/OOD_NLP}.Comment: Accepted to NeurIPS 2023 Dataset and Benchmark Track. Code is available at \url{https://github.com/lifan-yuan/OOD_NLP

CJRC: A Reliable Human-Annotated Benchmark DataSet for Chinese Judicial Reading Comprehension

We present a Chinese judicial reading comprehension (CJRC) dataset which contains approximately 10K documents and almost 50K questions with answers. The documents come from judgment documents and the questions are annotated by law experts. The CJRC dataset can help researchers extract elements by reading comprehension technology. Element extraction is an important task in the legal field. However, it is difficult to predefine the element types completely due to the diversity of document types and causes of action. By contrast, machine reading comprehension technology can quickly extract elements by answering various questions from the long document. We build two strong baseline models based on BERT and BiDAF. The experimental results show that there is enough space for improvement compared to human annotators

An improved model for predicting effective Young's modulus of the twisted structure under cyclic loading: taking into account the untwisting effect

ABSTRACT: Twist structures have diverse applications, ranging from dragline, electrical cable, and intelligent structure. Among these applications, tension deformation can't be avoided during the fabrication and working processes, which often leads to the twist structure rotation (called untwisting effect) and twist pitch increasing. As a consequence, this untwisting behavior has a large effect on the effective Young's modulus. In this paper, we present an improved model based on the classical Costello's theory to predict the effective Young's modulus of the basic structure, twisted by three same copper strands under cyclic loading. Series of experiments were carried out to verify the present model taking into account the untwisting effect. The experimental results have better agreements with the presented model than the common Costello's model. Keywords: Twist structure, Young's modulus, Cyclic loading, Untwisting effec

Inositol Hexaphosphate and Inositol Inhibit Colorectal Cancer Metastasis to the Liver in BALB/c Mice

Inositol hexaphosphate (IP6) and inositol (Ins), naturally occurring carbohydrates present in most mammals and plants, inhibit the growth of numerous cancers both in vitro and in vivo. In this study, we first examined the anti-metastatic effects of IP6 and Ins using a liver metastasis model of colorectal cancer (CRC) in BALB/c mice. CT-26 cells were injected into the splenic capsule of 48 BALB/c mice. The mice were then randomly divided into four groups: IP6, Ins, IP6 + Ins and normal saline control (n = 12 per group). IP6 and/or Ins (80 mg/kg each, 0.2 mL/day) were injected into the gastrointestinal tracts of the mice on the second day after surgery. All mice were sacrificed after 20 days, and the tumor inhibition rates were determined. The results demonstrated that the tumor weights of liver metastases and the tumor inhibition rates were reduced in the experimental groups compared to the control group and that treatment with the combination of IP6 and Ins resulted in greater inhibition of tumor growth than treatment with either compound alone. These findings suggest that IP6 and Ins prevent the development and metastatic progression of colorectal cancer to the liver in mice by altering expression of the extracellular matrix proteins collagen IV, fibronectin and laminin; the adhesion factor receptor integrin-β1; the proteolytic enzyme matrix metalloproteinase 9; and the angiogenic factors vascular endothelial growth factor, basic fibroblast growth factor, and transforming growth factor beta in the tumor metastasis microenvironment. In conclusion, IP6 and Ins inhibited the development and metastatic progression of colorectal cancer to the liver in BALB/c mice, and the effect of their combined application was significantly greater than the effect of either compound alone. This evidence supports further testing of the combined application of IP6 and Ins for the prevention of colorectal cancer metastasis to the liver in clinical studies

Large-Strain Nonlinear Consolidation of Sand-Drained Foundations Considering Vacuum Preloading and the Variation in Radial Permeability Coefficient

The vacuum preloading method effectively strengthens soft soil foundations with vertical drainage, which produces a smear effect when laying sand drains. Meanwhile, the seepage of pore water and soil deformation during consolidation exhibit nonlinear characteristics. Therefore, based on Gibson’s 1D large-strain consolidation theory, this paper developed a more generalized large-strain radical consolidation model of sand-drained soft foundations under free-strain assumptions. In this system, the double logarithmic compression permeability relationships for soft soils with large-strain properties, the variation in the radical permeability coefficient in the smear zone, and the effect of the non-Darcy flow were all included. Then, the partial differential control equations were numerically solved by the finite difference method and validated with existing radical consolidation test results and derived analytical solutions. Finally, the influences of relevant model parameters on consolidation are discussed. The analysis shows that the greater the maximum dimensionless vacuum negative pressure P0, the faster the consolidation rate of sand-drained foundations. Meanwhile, the decrease in the negative pressure transfer coefficient k1 will result in a decreasing final settlement amount. Moreover, the consolidation rate of sand-drained foundations is slower considering the non-Darcy flow, but the final settlement is unaffected

Swainsonine represses glioma cell proliferation, migration and invasion by reduction of miR-92a expression

Abstract Background Swainsonine is a natural indolizidine alkaloid, its anti-tumor activity has been widely reported in varied cancers. This study aimed to investigate whether Swainsonine exerted anti-tumor impact on glioma cells, likewise uncovered the relative molecular mechanisms. Methods After administration with diverse concentrations of Swainsonine, cell growth, migration and invasion in U251 and LN444 cells were appraised by the common-used CCK-8, BrdU, flow cytometry and Transwell assays. MiR-92a mimic, inhibitor and the correlative NC were transfected into U251 and LN444 cells, and assessment of miR-92a expression was by utilizing qRT-PCR. Functions of miR-92a in above-mentioned cell biological processes were analyzed again in Swainsonine-treated cells. The momentous proteins of cell cycle, apoptosis and PI3K/AKT/mTOR pathway were ultimately examined by western blot. Results Swainsonine significantly hindered cell proliferation through decreasing cell viability, declining the percentage of BrdU cells, down-regulating CyclinD1 and up-regulating p16 expression. Enhancement of percentage of apoptotic cells was presented in Swainsonine-treated cells via activating cleaved-Caspase-3 and cleaved-Caspase-9. Additionally, Swainsonine impeded the abilities of migration and invasion by decreasing MMP-2, MMP-9, Vimentin and E-cadherin. Repression of miR-92a was observed in Swainsonine-treated cells, and miR-92a overexpression overturned the anti-tumor activity of Swainsonine in glioma cells. Finally, western blot assay displayed that Swainsonine hindered PI3K/AKT/mTOR pathway via regulating miR-92a. Conclusions These discoveries corroborated that Swainsonine exerted anti-tumor impacts on glioma cells via repression of miR-92a, and inactivation of PI3K/AKT/mTOR signaling pathway

Femoral Neck System versus Three Cannulated Screws for Fixation of Femoral Neck Fractures in Younger Patients: A Retrospective Cohort Study

Objective To compare the clinical results of a new femoral neck system (FNS) and cannulated compression screws (CCS) for the treatment of femoral neck fractures in younger patients. Methods Retrospective study was performed in younger patients with femoral neck fractures that were treated with FNS or CCS from August 2017 to August 2022. The hip functional outcomes were assessed with the Harris hip score (HHS). Secondary outcomes included the surgical time, surgical blood loss, satisfaction visual analog scale (VAS), fluoroscopy frequency, fracture healing time and complications. Results A total of 49 patients (22 FNS and 27 CCS) with a minimum follow-up of 12 months were included. There was also no significant difference in HHSs (p = 0.27) and satisfaction VAS (p = 0.10) between them. Patients with FNS had more blood loss (50.45 ± 5.28 mL vs. 20.67 ± 4.71 ml, p < 0.01), lower fluoroscopy frequency (16.64 ± 3.32 vs. 23.59 ± 3.39, p < 0.01) and lower fracture healing time (3.76 ± 0.42 vs. 4.46 ± 0.59 months, p < 0.01). The femoral neck shortening and incidence of nail withdrawal in the FNS group was significantly lower than CCS group (2.91 ± 1.95 vs. 4.44 ± 1.52 mm, p < 0.01; 4.5% vs. 22.2%, p = 0.03). Conclusions The FNS and CCS get similar hip functions. The FNS can reduce fluoroscopy exposure and the complications such as femoral neck shortening and nail withdrawal. Thus, FNS can be an alternative to CCS for the fixation of femoral neck fractures in younger patients