FedHyper: A Universal and Robust Learning Rate Scheduler for Federated Learning with Hypergradient Descent

The theoretical landscape of federated learning (FL) undergoes rapid evolution, but its practical application encounters a series of intricate challenges, and hyperparameter optimization is one of these critical challenges. Amongst the diverse adjustments in hyperparameters, the adaptation of the learning rate emerges as a crucial component, holding the promise of significantly enhancing the efficacy of FL systems. In response to this critical need, this paper presents FedHyper, a novel hypergradient-based learning rate adaptation algorithm specifically designed for FL. FedHyper serves as a universal learning rate scheduler that can adapt both global and local rates as the training progresses. In addition, FedHyper not only showcases unparalleled robustness to a spectrum of initial learning rate configurations but also significantly alleviates the necessity for laborious empirical learning rate adjustments. We provide a comprehensive theoretical analysis of FedHyper's convergence rate and conduct extensive experiments on vision and language benchmark datasets. The results demonstrate that FEDHYPER consistently converges 1.1-3x faster than FedAvg and the competing baselines while achieving superior final accuracy. Moreover, FedHyper catalyzes a remarkable surge in accuracy, augmenting it by up to 15% compared to FedAvg under suboptimal initial learning rate settings

VT-CLIP: Enhancing Vision-Language Models with Visual-guided Texts

Contrastive Language-Image Pre-training (CLIP) has drawn increasing attention recently for its transferable visual representation learning. However, due to the semantic gap within datasets, CLIP's pre-trained image-text alignment becomes sub-optimal on downstream tasks, which severely harms its transferring performance. To better adapt the cross-modality embedding space, we propose to enhance CLIP via Visual-guided Texts, named VT-CLIP. Specifically, we guide textual features of different categories to adaptively explore informative regions on the image and aggregate visual features by attention mechanisms. In this way, the texts become visual-guided, namely, more semantically correlated with downstream images, which greatly benefits the category-wise matching process. In few-shot settings, we evaluate our VT-CLIP on 11 well-known classification datasets to demonstrate its effectiveness

GaitFormer: Revisiting Intrinsic Periodicity for Gait Recognition

Gait recognition aims to distinguish different walking patterns by analyzing video-level human silhouettes, rather than relying on appearance information. Previous research on gait recognition has primarily focused on extracting local or global spatial-temporal representations, while overlooking the intrinsic periodic features of gait sequences, which, when fully utilized, can significantly enhance performance. In this work, we propose a plug-and-play strategy, called Temporal Periodic Alignment (TPA), which leverages the periodic nature and fine-grained temporal dependencies of gait patterns. The TPA strategy comprises two key components. The first component is Adaptive Fourier-transform Position Encoding (AFPE), which adaptively converts features and discrete-time signals into embeddings that are sensitive to periodic walking patterns. The second component is the Temporal Aggregation Module (TAM), which separates embeddings into trend and seasonal components, and extracts meaningful temporal correlations to identify primary components, while filtering out random noise. We present a simple and effective baseline method for gait recognition, based on the TPA strategy. Extensive experiments conducted on three popular public datasets (CASIA-B, OU-MVLP, and GREW) demonstrate that our proposed method achieves state-of-the-art performance on multiple benchmark tests

SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device

With the rapid development of AI hardware accelerators, applying deep learning-based algorithms to solve various low-level vision tasks on mobile devices has gradually become possible. However, two main problems still need to be solved: task-specific algorithms make it difficult to integrate them into a single neural network architecture, and large amounts of parameters make it difficult to achieve real-time inference. To tackle these problems, we propose a novel network, SYENet, with only $~$6K parameters, to handle multiple low-level vision tasks on mobile devices in a real-time manner. The SYENet consists of two asymmetrical branches with simple building blocks. To effectively connect the results by asymmetrical branches, a Quadratic Connection Unit(QCU) is proposed. Furthermore, to improve performance, a new Outlier-Aware Loss is proposed to process the image. The proposed method proves its superior performance with the best PSNR as compared with other networks in real-time applications such as Image Signal Processing(ISP), Low-Light Enhancement(LLE), and Super-Resolution(SR) with 2K60FPS throughput on Qualcomm 8 Gen 1 mobile SoC(System-on-Chip). Particularly, for ISP task, SYENet got the highest score in MAI 2022 Learned Smartphone ISP challenge

Effect of BRCA1 R1325K mutation on proliferation and apoptosis of gallbladder cancer cells

Objective·To investigate the effects of breast cancer susceptibility gene 1 (BRCA1) R1325K mutation [arginine (R) to lysine (K) mutation at amino acid 1325] on the proliferation and apoptosis of gallbladder cancer cell lines GBC-SD and NOZ.Methods·BRCA1 wild-type overexpression lentivirus, BRCA1 R1325K mutation overexpression lentivirus, and negative control lentivirus were used to construct the stable transgenic strains of gallbladder carcinoma, cell lines GBC-SD and NOZ. The cells were divided into the control group without the target gene, the BRCA1 wild-type group, and the BRCA1 R1325K mutation group. The expression of target protein was verified by Western blotting. The BRCA1 R1325K mutant gallbladder cancer cells were treated with 20 μmol/L Olaparib, a BRCA1 mutation inhibitor. Gallbladder cancer cell lines were divided into the control group, the BRCA1 wild-type group, the BRCA1 R1325K mutation group, and the BRCA1 R1325K mutation+Olaparib group according to the target gene expression and whether or not the inhibitor was added. The effect of BRCA1 R1325K mutation on proliferation and clonogenesis ability of gallbladder cancer cell lines GBC-SD and NOZ was observed by CCK8 assay and clonogenesis assay, respectively. The effect of BRCA1 R1325K mutation on apoptosis of gallbladder cancer cell lines GBC-SD and NOZ was observed by TUNEL assay. The expressions of apoptosis-related proteins, cleaved PARP, Bcl-2 and Bax, were detected by Western blotting. The inhibitor Olaparib was used to treat the BRCA1 R1325K mutant gallbladder cancer cell lines GBC-SD and NOZ. The phenotypic changes (promoting proliferation, enhancing clonogenesis and inhibiting apoptosis) induced by BRCA1 R1325K mutation were tested in the presence of Olaparib to determine whether the changes could be reversed by the inhibitor.Results·The results of CCK8 assay and clonogenesis assay showed that BRCA1 R1325K mutation could promote the proliferation of gallbladder cancer cell lines GBC-SD and NOZ, and improve their clonal formation ability, compared with the control group and the BRCA1 wild-type group. Olaparib inhibited the proliferation of gallbladder cancer cell lines overexpressing BRCA1 R1325K mutation (P<0.05). Through TUNEL and Western blotting, it was found that overexpression of wild-type BRCA1 could induce the apoptosis of gallbladder cancer cell lines GBC-SD and NOZ, compared with the control group. Compared with the control group and the BRCA1 wild-type group, the BRCA1 R1325K mutation group had anti-apoptotic effect, in which the expression of apoptosis-inhibiting protein Bcl-2 increased and the expression of pro-apoptotic protein Bax decreased (P<0.05).Conclusion·BRCA1 R1325K mutation can promote the proliferation of GBC-SD and NOZ cell lines and inhibit their apoptosis

Vancomycin efficiency and safety of a dosage of 40–60 mg/kg/d and corresponding trough concentrations in children with Gram-positive bacterial sepsis

BackgroundOptimal vancomycin trough concentrations and dosages remain controversial in sepsis children. We aim to investigate vancomycin treatment outcomes with a dosage of 40-60 mg/kg/d and corresponding trough concentrations in children with Gram-positive bacterial sepsis from a clinical perspective.MethodsChildren diagnosed with Gram-positive bacterial sepsis and received intravenous vancomycin therapy between January 2017 and June 2020 were enrolled retrospectively. Patients were categorized as success and failure groups according to treatment outcomes. Laboratory, microbiological, and clinical data were collected. The risk factors for treatment failure were analyzed by logistic regression.ResultsIn total, 186 children were included, of whom 167 (89.8%) were enrolled in the success group and 19 (10.2%) in the failure group. The initial and mean vancomycin daily doses in failure group were significantly higher than those in success group [56.9 (IQR =42.1-60.0) vs. 40.5 (IQR =40.0-57.1), P=0.016; 57.0 (IQR =45.8-60.0) vs. 50.0 (IQR =40.0-57.6) mg/kg/d, P=0.012, respectively] and median vancomycin trough concentrations were similar between two groups [6.9 (4.0-12.1) vs.7.3 (4.5-10.6) mg/L, P=0.568)]. Moreover, there was no significant differences in treatment success rate between vancomycin trough concentrations ≤15 mg/L and &gt;15 mg/L (91.2% vs. 75.0%, P=0.064). No vancomycin-related nephrotoxicity adverse effects occurred among all enrolled patients. Multivariate analysis revealed that a PRISM III score ≥10 (OR =15.011; 95% CI: 3.937-57.230; P&lt;0.001) was the only independent clinical factor associated with increased incidence of treatment failure.ConclusionsVancomycin dosages of 40-60 mg/kg/d are effective and have no vancomycin-related nephrotoxicity adverse effects in children with Gram-positive bacterial sepsis. Vancomycin trough concentrations &gt;15 mg/L are not an essential target for these Gram-positive bacterial sepsis patients. PRISM III scores ≥10 may serve as an independent risk factor for vancomycin treatment failure in these patients