A Comprehensive Empirical Study of Bugs in Open-Source Federated Learning Frameworks

Federated learning (FL) is a distributed machine learning (ML) paradigm, allowing multiple clients to collaboratively train shared machine learning (ML) models without exposing clients' data privacy. It has gained substantial popularity in recent years, especially since the enforcement of data protection laws and regulations in many countries. To foster the application of FL, a variety of FL frameworks have been proposed, allowing non-experts to easily train ML models. As a result, understanding bugs in FL frameworks is critical for facilitating the development of better FL frameworks and potentially encouraging the development of bug detection, localization and repair tools. Thus, we conduct the first empirical study to comprehensively collect, taxonomize, and characterize bugs in FL frameworks. Specifically, we manually collect and classify 1,119 bugs from all the 676 closed issues and 514 merged pull requests in 17 popular and representative open-source FL frameworks on GitHub. We propose a classification of those bugs into 12 bug symptoms, 12 root causes, and 18 fix patterns. We also study their correlations and distributions on 23 functionalities. We identify nine major findings from our study, discuss their implications and future research directions based on our findings

Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment

With the continuous growth in the number of parameters of transformer-based pretrained language models (PLMs), particularly the emergence of large language models (LLMs) with billions of parameters, many natural language processing (NLP) tasks have demonstrated remarkable success. However, the enormous size and computational demands of these models pose significant challenges for adapting them to specific downstream tasks, especially in environments with limited computational resources. Parameter Efficient Fine-Tuning (PEFT) offers an effective solution by reducing the number of fine-tuning parameters and memory usage while achieving comparable performance to full fine-tuning. The demands for fine-tuning PLMs, especially LLMs, have led to a surge in the development of PEFT methods, as depicted in Fig. 1. In this paper, we present a comprehensive and systematic review of PEFT methods for PLMs. We summarize these PEFT methods, discuss their applications, and outline future directions. Furthermore, we conduct experiments using several representative PEFT methods to better understand their effectiveness in parameter efficiency and memory efficiency. By offering insights into the latest advancements and practical applications, this survey serves as an invaluable resource for researchers and practitioners seeking to navigate the challenges and opportunities presented by PEFT in the context of PLMs.Comment: 20 pages, 4 figure

Prevalent Exon-Intron Structural Changes in the APETALA1/FRUITFULL, SEPALLATA, AGAMOUS-LIKE6, and FLOWERING LOCUS C MADS-Box Gene Subfamilies Provide New Insights into Their Evolution

AP1/FUL, SEP, AGL6, and FLC subfamily genes play important roles in flower development. The phylogenetic relationships among them, however, have been controversial, which impedes our understanding of the origin and functional divergence of these genes. One possible reason for the controversy may be the problems caused by changes in the exon-intron structure of genes, which, according to recent studies, may generate non-homologous sites and hamper the homology-based sequence alignment. In this study, we first performed exon-by-exon alignments of these and three outgroup subfamilies (SOC1, AG, and STK). Phylogenetic trees reconstructed based on these matrices show improved resolution and better congruence with species phylogeny. In the context of these phylogenies, we traced evolutionary changes of exon-intron structures in each subfamily. We found that structural changes have occurred frequently following gene duplication and speciation events. Notably, exons 7 and 8 (if present) suffered more structural changes than others. With the knowledge of exon-intron structural changes, we generated more reasonable alignments containing all the focal subfamilies. The resulting trees showed that the SEP subfamily is sister to the monophyletic group formed by AP1/FUL and FLC subfamily genes and that the AGL6 subfamily forms a sister group to the three abovementioned subfamilies. Based on this topology, we inferred the evolutionary history of exon-intron structural changes among different subfamilies. Particularly, we found that the eighth exon originated before the divergence of AP1/FUL, FLC, SEP, and AGL6 subfamilies and degenerated in the ancestral FLC-like gene. These results provide new insights into the origin and evolution of the AP1/FUL, FLC, SEP, and AGL6 subfamilies

Generation of Kerr soliton microcomb in a normally dispersed lithium niobate microdisk resonator by mode trimming

Anomalous microresonator dispersion is mandatory for Kerr soliton microcomb formation, which depends critically on the geometry of the microresonator and can hardly be tuned after the structure is made. To date, cavity-based microcombs have only been generated with fundamental whispering gallery modes (WGMs) of anomalous dispersion in microresonators. Moreover, microcomb generation in highly Raman-active platforms such as lithium niobate (LN) microresonators frequently suffers from stimulated Raman scattering and mode crossing due to the existence of multiple families of high-order WGMs. Here, we reveal a unique Kerr soliton microcomb generation mechanism through mode trimming in a weakly perturbed LN microdisk resonator. Remarkably, the soliton comb is generated with fundamental WGMs of normal dispersion and free from the mode crossing and Raman scattering effects. A robust soliton with a spectrum spanning from 1450 nm to 1620 nm at an on-chip pump power of 35 mW. Our discovery offers a powerful solution to circumvent the stringent requirements on high-precision dispersion engineering and termination of Raman excitation for soliton generation in the high-Q microdisk.Comment: 16 pages,and 5 figure

Folate receptor-targeted mixed polysialic acid micelles for combating rheumatoid arthritis: in vitro and in vivo evaluation

Objective: Rheumatoid arthritis (RA) is associated with chronic inflammation. The suppression of inflammation is key to the treatment of RA. Glucocorticoids (GCs) are classical anti-inflammatory drugs with several disadvantages such as poor water solubility and low specificity in the body. These disadvantages are the reasons for the quick elimination and side effects of GCs in vivo. Micelles are ideal carriers for GCs delivery to inflamed synovium. We set out to improve the targeting and pharmacokinetic profiles of GCs by preparing a targeting micelle system. Methods: In this study, natural chlosterol (CC) and folic acid (FA) were used to fabricate polysialic acid (PSA) micelles for the targeted delivery of Dexamethasone (Dex). The biodistribution and therapeutic efficacy of the resulting micelles were evaluated in vitro and in vivo. Results: PSA-CC and FA-PSA-CC micelles showed a size below 100 nm and a moderate negative charge. PSA-CC and FA-PSA-CC micelles could also enhance the intracellular uptake of Dex and the suppression of tumor necrosis factor-α (TNF-α) and interleukin-6 (IL-6) in vitro and in vivo. Arthritis mice showed reduced paw thickness and clinical arthritis index using PSA-CC and FA-PSA-CC micelle treatment. Micellized Dex demonstrated a 4 ∼ 5 fold longer elimination half-life and a 2 ∼ 3 folds higher bioavailability than commercial Dex injection. FA modification significantly improved the anti-inflammatory efficacy of PSA-CC micelles. Conclusion: FA-PSA-CC micelles demonstrated significant advantages in terms of the suppression of inflammation and the treatment of inflammatory arthritis. These reliable and stable micelles possess a high potential to be transferred for clinical use

Oil absorption stability of modified cellulose porous materials with super compressive strength in the complex environment

The occurrence of oil spills has severe damage upon both the environment and human health. Hence, the development of a green, recyclable, complex environment resistant, and efficient oil–water separation aerogel is required in order to effectively absorb marine or industrial oil. In this study, modified cellulose/N,N'-methylenebisacrylamide/tannin (PCMT) composite porous materials were prepared utilizing the sol–gel method and were modified with tertbutyl acrylate. PCMT possesses a three-dimensional interpenetrating porous structure, exhibiting remarkable oil–water separation performance and excellent compressive strength (PCMT can capable of bearing 7000 times its own weight; PCMT can endure 290.3 kPa pressure at 80% strain when the amount of tannin is 0.2 g). The unique pore structure of PCMT engenders differential oil adsorption capacities (PCMT0, PCMT0.05, PCMT0.1, and PCMT0.2 evince higher adsorption capacities for petroleum ether and dichloromethane, n-hexane and dichloromethane, toluene, and toluene and dichloromethane, respectively). Of critical import, PCMT demonstrates exceptional adaptability to complex environments, wherein the porous materials maintain good hydrophobicity and oil absorption capacity under conditions of vigorous stirring, a wide pH range (1–14), a wide temperature range (4–160 °C), ultraviolet irradiation (8 h), and tape peeling (10 times). Moreover, the porous materials may be employed for the recovery of oil through simple mechanical extrusion, thus demonstrating certain economic significance and the application potential in the treatment of oil spills.publishedVersio