A Comprehensive Empirical Study of Bugs in Open-Source Federated Learning Frameworks

Federated learning (FL) is a distributed machine learning (ML) paradigm, allowing multiple clients to collaboratively train shared machine learning (ML) models without exposing clients' data privacy. It has gained substantial popularity in recent years, especially since the enforcement of data protection laws and regulations in many countries. To foster the application of FL, a variety of FL frameworks have been proposed, allowing non-experts to easily train ML models. As a result, understanding bugs in FL frameworks is critical for facilitating the development of better FL frameworks and potentially encouraging the development of bug detection, localization and repair tools. Thus, we conduct the first empirical study to comprehensively collect, taxonomize, and characterize bugs in FL frameworks. Specifically, we manually collect and classify 1,119 bugs from all the 676 closed issues and 514 merged pull requests in 17 popular and representative open-source FL frameworks on GitHub. We propose a classification of those bugs into 12 bug symptoms, 12 root causes, and 18 fix patterns. We also study their correlations and distributions on 23 functionalities. We identify nine major findings from our study, discuss their implications and future research directions based on our findings

Revisit Parameter-Efficient Transfer Learning: A Two-Stage Paradigm

Parameter-Efficient Transfer Learning (PETL) aims at efficiently adapting large models pre-trained on massive data to downstream tasks with limited task-specific data. In view of the practicality of PETL, previous works focus on tuning a small set of parameters for each downstream task in an end-to-end manner while rarely considering the task distribution shift issue between the pre-training task and the downstream task. This paper proposes a novel two-stage paradigm, where the pre-trained model is first aligned to the target distribution. Then the task-relevant information is leveraged for effective adaptation. Specifically, the first stage narrows the task distribution shift by tuning the scale and shift in the LayerNorm layers. In the second stage, to efficiently learn the task-relevant information, we propose a Taylor expansion-based importance score to identify task-relevant channels for the downstream task and then only tune such a small portion of channels, making the adaptation to be parameter-efficient. Overall, we present a promising new direction for PETL, and the proposed paradigm achieves state-of-the-art performance on the average accuracy of 19 downstream tasks.Comment: 11 page

SCT: A Simple Baseline for Parameter-Efficient Fine-Tuning via Salient Channels

Pre-trained vision transformers have strong representation benefits to various downstream tasks. Recently, many parameter-efficient fine-tuning (PEFT) methods have been proposed, and their experiments demonstrate that tuning only 1% of extra parameters could surpass full fine-tuning in low-data resource scenarios. However, these methods overlook the task-specific information when fine-tuning diverse downstream tasks. In this paper, we propose a simple yet effective method called "Salient Channel Tuning" (SCT) to leverage the task-specific information by forwarding the model with the task images to select partial channels in a feature map that enables us to tune only 1/8 channels leading to significantly lower parameter costs. Experiments outperform full fine-tuning on 18 out of 19 tasks in the VTAB-1K benchmark by adding only 0.11M parameters of the ViT-B, which is 780$\times$ fewer than its full fine-tuning counterpart. Furthermore, experiments on domain generalization and few-shot learning surpass other PEFT methods with lower parameter costs, demonstrating our proposed tuning technique's strong capability and effectiveness in the low-data regime.Comment: This work has been accepted by IJCV202

Comparison of Different Transfer Learning Methods for Classification of Mangrove Communities Using MCCUNet and UAV Multispectral Images

Mangrove-forest classification by using deep learning algorithms has attracted increasing attention but remains challenging. The current studies on the transfer classification of mangrove communities between different regions and different sensors are especially still unclear. To fill the research gap, this study developed a new deep-learning algorithm (encoder–decoder with mixed depth-wise convolution and cascade upsampling, MCCUNet) by modifying the encoder and decoder sections of the DeepLabV3+ algorithm and presented three transfer-learning strategies, namely frozen transfer learning (F-TL), fine-tuned transfer learning (Ft-TL), and sensor-and-phase transfer learning (SaP-TL), to classify mangrove communities by using the MCCUNet algorithm and high-resolution UAV multispectral images. This study combined the deep-learning algorithms with recursive feature elimination and principal component analysis (RFE–PCA), using a high-dimensional dataset to map and classify mangrove communities, and evaluated their classification performance. The results of this study showed the following: (1) The MCCUNet algorithm outperformed the original DeepLabV3+ algorithm for classifying mangrove communities, achieving the highest overall classification accuracy (OA), i.e., 97.24%, in all scenarios. (2) The RFE–PCA dimension reduction improved the classification performance of deep-learning algorithms. The OA of mangrove species from using the MCCUNet algorithm was improved by 7.27% after adding dimension-reduced texture features and vegetation indices. (3) The Ft-TL strategy enabled the algorithm to achieve better classification accuracy and stability than the F-TL strategy. The highest improvement in the F1–score of Spartina alterniflora was 19.56%, using the MCCUNet algorithm with the Ft-TL strategy. (4) The SaP-TL strategy produced better transfer-learning classifications of mangrove communities between images of different phases and sensors. The highest improvement in the F1–score of Aegiceras corniculatum was 19.85%, using the MCCUNet algorithm with the SaP-TL strategy. (5) All three transfer-learning strategies achieved high accuracy in classifying mangrove communities, with the mean F1–score of 84.37~95.25%

16S Next-generation sequencing and quantitative PCR reveal the distribution of potential pathogens in the Liaohe Estuary

The existence of potentially pathogenic bacteria seriously threatens aquatic animals and human health. Estuaries are closely related to human activities, and the detection of pathogens is important for aquaculture and public health. However, monitoring only indicator microorganisms and pathogens is not enough to accurately and comprehensively estimate water pollution. Here, the diversity of potentially pathogenic bacteria in water samples from the Liaohe estuary was profiled using 16S next-generation sequencing (16S NGS) and quantitative polymerase chain reaction (qPCR) analysis. The results showed that the dominant genera of environmental pathogens were Pseudomonas, Vibrio, Mycobacterium, Acinetobacter, Exiguobacterium, Sphingomonas, and Legionella, and the abundance of enteric pathogens was significantly less than the environmental pathogens, mainly, Citrobacter, Enterococcus, Escherichia-Shigella, Enterobacter, Bacteroides. The qPCR results showed that the 16S rRNA genes of Vibrio were the most abundant, with concentrations between 7.06 and 9.48 lg copies/L, followed by oaa gene, fliC gene, trh gene, and uidA gene, and the temperature and salinity were the main factors affecting its abundance. Variance partitioning analysis (VPA) analysis of spatial factors on the potential pathogen’s distribution (19.6% vs 5.3%) was greater than environmental factors. In addition, the co-occurrence analysis of potential pathogens in the estuary revealed significant co-occurrence among the opportunistic pathogens Testosteronemonas, Brevimonas vesicularis, and Pseudomonas putida. Our findings provide an essential reference for monitoring and occurrence of potentially pathogenic bacteria in estuaries

MicroRNA-483 amelioration of experimental pulmonary hypertension.

Endothelial dysfunction is critically involved in the pathogenesis of pulmonary arterial hypertension (PAH) and that exogenously administered microRNA may be of therapeutic benefit. Lower levels of miR-483 were found in serum from patients with idiopathic pulmonary arterial hypertension (IPAH), particularly those with more severe disease. RNA-seq and bioinformatics analyses showed that miR-483 targets several PAH-related genes, including transforming growth factor-β (TGF-β), TGF-β receptor 2 (TGFBR2), β-catenin, connective tissue growth factor (CTGF), interleukin-1β (IL-1β), and endothelin-1 (ET-1). Overexpression of miR-483 in ECs inhibited inflammatory and fibrogenic responses, revealed by the decreased expression of TGF-β, TGFBR2, β-catenin, CTGF, IL-1β, and ET-1. In contrast, inhibition of miR-483 increased these genes in ECs. Rats with EC-specific miR-483 overexpression exhibited ameliorated pulmonary hypertension (PH) and reduced right ventricular hypertrophy on challenge with monocrotaline (MCT) or Sugen + hypoxia. A reversal effect was observed in rats that received MCT with inhaled lentivirus overexpressing miR-483. These results indicate that PAH is associated with a reduced level of miR-483 and that miR-483 might reduce experimental PH by inhibition of multiple adverse responses

Microwave-assisted aqueous two-phase extraction of alkaloids from Radix Sophorae Tonkinensis with ethanol/Na2HPO4 system: process optimization, composition identification and quantification analysis

A rapid method for simultaneous extraction and separation of multiple alkaloids from Radix Sophorae Tonkinensis (RST) was developed by microwave-assisted aqueous two-phase extraction (MAATPE) using the aqueous two-phase extraction system (ATPS) of ethanol/Na2HPO4 as the extraction solvent. The effects of key factors on extraction yield were investigated by utilizing single-factor experiment coupled to response surface methodology (RSM). The regression model by RSM was significant (p < 0.0001) and adequate for prediction of process efficacy, the optimized conditions were successfully validated by the parallel experiments with the yield very close to the predicted value. The optimum conditions were summarized as follows: the phase ratio of 2.60 for the ATPS, the particle size of 100 mesh, the liquid-to-material ratio of 75:1, the extraction temperature of 90 °C and the extraction time of 5 min, respectively. In MAATPE process, alkaloids were extracted preferentially from RST in the top phase with a higher yield and shorter extraction time than those of heating reflux extraction (HRE) and ultrasonic-assisted extraction (UAE). Nine alkaloids extracted were identified and quantified by high-resolution ultra-performance liquid chromatography-quadrupole-orbitrap mass spectrometry (UPLC-Q-Orbitrap/MS) and HPLC with UV detection. The contents of matrine, sophocarpine, oxymatrin, sophoranol, oxysophocarpine, 5α-hydroxysophocarpine, sophoridine, cytisine and N-methylcytisine in RST were quantified in range of 0.493–10.284 mg/g with recoveries of 90.26–106.3% and RSD’s of 0.8–2.1%, respectively. Moreover, the MAATPE mechanism was explored using the different extraction systems in combination of scanning electron microscopy (SEM) morphological studies. Significant differences in extraction yield and cell rupture exhibited that the addition of the salt in the ethanol-water mixture not only improved the thermal and demixing effects, but also accelerated the mass transfer and biphasic extraction processes. MAATPE integrated the advantages of microwave-assisted extraction (MAE) and aqueous two-phase extraction (ATPE) was proved as a green, efficient and promising alternative to extraction of alkaloids from RST