Personalized Federated Learning via ADMM with Moreau Envelope

Personalized federated learning (PFL) is an approach proposed to address the issue of poor convergence on heterogeneous data. However, most existing PFL frameworks require strong assumptions for convergence. In this paper, we propose an alternating direction method of multipliers (ADMM) for training PFL models with Moreau envelope (FLAME), which achieves a sublinear convergence rate, relying on the relatively weak assumption of gradient Lipschitz continuity. Moreover, due to the gradient-free nature of ADMM, FLAME alleviates the need for hyperparameter tuning, particularly in avoiding the adjustment of the learning rate when training the global model. In addition, we propose a biased client selection strategy to expedite the convergence of training of PFL models. Our theoretical analysis establishes the global convergence under both unbiased and biased client selection strategies. Our experiments validate that FLAME, when trained on heterogeneous data, outperforms state-of-the-art methods in terms of model performance. Regarding communication efficiency, it exhibits an average speedup of 3.75x compared to the baselines. Furthermore, experimental results validate that the biased client selection strategy speeds up the convergence of both personalized and global models.Comment: 15 page

Some New Inequalities of Jordan Type for Sine

The authors find some new inequalities of Jordan type for the sine function. These newly established inequalities are of new form and are applied to deduce some known results

IDa-Det: An Information Discrepancy-aware Distillation for 1-bit Detectors

Knowledge distillation (KD) has been proven to be useful for training compact object detection models. However, we observe that KD is often effective when the teacher model and student counterpart share similar proposal information. This explains why existing KD methods are less effective for 1-bit detectors, caused by a significant information discrepancy between the real-valued teacher and the 1-bit student. This paper presents an Information Discrepancy-aware strategy (IDa-Det) to distill 1-bit detectors that can effectively eliminate information discrepancies and significantly reduce the performance gap between a 1-bit detector and its real-valued counterpart. We formulate the distillation process as a bi-level optimization formulation. At the inner level, we select the representative proposals with maximum information discrepancy. We then introduce a novel entropy distillation loss to reduce the disparity based on the selected proposals. Extensive experiments demonstrate IDa-Det's superiority over state-of-the-art 1-bit detectors and KD methods on both PASCAL VOC and COCO datasets. IDa-Det achieves a 76.9% mAP for a 1-bit Faster-RCNN with ResNet-18 backbone. Our code is open-sourced on https://github.com/SteveTsui/IDa-Det

Retinal nerve fiber layer thickness measured by optical coherence tomography in Chinese teenagers aged from 13 years old to 18 years old

AIM: To establish a reference range of retinal nerve fiber layer(RNFL)thickness by optical coherence tomography(OCT)in Chinese teenagers aged from 13 to 18 years old, and investigate its relationship with age, eye side, gender, and ethnic group.<p>METHODS: A total of 402 eyes from 201 normal Chinese aged from 13 to 18 years old were recruited for this study. Optic disk with 3.4mm diameter circle in different global average, quadrant and part-time bit retinal nerve fiber layer thickness(RNFLT)was measured by RNFL thickness average analysis program. Their RNFLT at different part-time bit, quadrant and global average RNFLT around the disc were measured by OCT with 3.4mm diameter circle, using the RNFL thickness average analysis program. The data was analyzed with SPSS statistical 19.0. The influences of several factors(such as age, eye side, gender, and ethnic group)on RNFLT were also analyzed.<p>RESULTS: The global average RNFLT at 13, 14, 15, 16, 17, 18 years old was 108.32±9.42μm, 109.23±9.67μm, 110.36±11.14μm, 111.27±10.21μm, 109.23±9.67μm, 112.11±8.83μm respectively. RNFLT of right eyes was 109.82±8.93μm and of left eyes was 110.33±9.89μm. All of the male's average RNFLT was 110.14±10.02μm, and all of the female's average RNFLT was 109.96±11.22μm. The average RNFLT of Han nationality was 110.22±9.31μm and of non-Han nationality was 109.87±8.65μm. The average RNFLT of all was 110.02±9.87μm, the RNFLT at the superior, nasal, inferior and temporal quadrant was 146.56 ±18.88μm, 76.49±13.28μm, 136.64±16.29μm, 82.01±12.55μm respectively. There was no significant difference in gender, eye side, and ethnic group(all <i>P</i><0.05), but the difference was significant compared with adult's data of the database. <p>CONCLUSION: This study has established a normal standard reference of RNFLT and its related indexes by OCT in Chinese teenagers aged 13-18 years old. Gender, age, eye side, and ethnic group have no effect on their RNFLT, which has significant difference with adult's data. And for the diagnosis and follow-up of diseases involving in RNFL changes, the adult's data can not be used

Non-intact zona improves development of murine preimplantation embryos transfected by an adenovirus vector

The present study explored whether embryos could be transfected by the adenovirus-vector if the zona pellucida (ZP) was not completely removed. An adenovirus vector with green fluorescent protein (pAd-GFP) was used to transfect mouse non-intact zona zygotes (following partial removal of the ZP induced by pronase), zona-free and zona-intact embryos. Non-intact zona and zona-free embryos expressed GFP (confirmed with inverted fluorescence microscopy) after 48 h of culture. The transfection rate of non-intact zona group was up to 51% and the entire zona-free group was transfected. However, none of the zona-intact embryos was transfected. Regardless of whether non-intact zona embryos were transfected by pAd-GFP, their developmental rate (74.3 ± 2.4 and 69.2 ± 3.3% for non-transfected and transfected, respectively; mean ± SEM) was higher (P&lt;0.05) than that of zona-free embryos without and with transfection (54.5 ± 4.3 and 46.7 ± 5.5%). Developmental potential of embryos was decreased for ZP-digestion (non-intact zona 71.8 ± 1.6%; zona-free 50.6 ± 2.2%, P&lt;0.05) or pAd-GFP expression (non-transfected 64.4 ± 1.9%; transfected 56.0 ± 2.1%, P&lt;0.05); therefore, ZP-digestion affected more intensely embryos development than pAd-GFP expression. In summary, non-intact zona murine embryos were readily transfected by the adenovirus-vector, and had much greater development potential than zona-free embryos. Although, the susceptibility of the ZP to digestion by pronase varied among embryos, on average, approximately 3.5 to 4.0 min of digestion resulted in partial removal of the ZP and promoted both transfection and satisfactory embryonic development. It is expected that this method could be used to increase the efficiency of generating transgenic animals.Keywords: Mouse, non-intact zona embryos, adenovirus vector with green fluorescent protein (pAd-GFP), embryos developmen