Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient

Over-the-air computation is a communication-efficient solution for federated learning (FL). In such a system, iterative procedure is performed: Local gradient of private loss function is updated, amplified and then transmitted by every mobile device; the server receives the aggregated gradient all-at-once, generates and then broadcasts updated model parameters to every mobile device. In terms of amplification factor selection, most related works suppose the local gradient's maximal norm always happens although it actually fluctuates over iterations, which may degrade convergence performance. To circumvent this problem, we propose to turn local gradient to be normalized one before amplifying it. Under our proposed method, when the loss function is smooth, we prove our proposed method can converge to stationary point at sub-linear rate. In case of smooth and strongly convex loss function, we prove our proposed method can achieve minimal training loss at linear rate with any small positive tolerance. Moreover, a tradeoff between convergence rate and the tolerance is discovered. To speedup convergence, problems optimizing system parameters are also formulated for above two cases. Although being non-convex, optimal solution with polynomial complexity of the formulated problems are derived. Experimental results show our proposed method can outperform benchmark methods on convergence performance

Federated Learning with Manifold Regularization and Normalized Update Reaggregation

Federated Learning (FL) is an emerging collaborative machine learning framework where multiple clients train the global model without sharing their own datasets. In FL, the model inconsistency caused by the local data heterogeneity across clients results in the near-orthogonality of client updates, which leads to the global update norm reduction and slows down the convergence. Most previous works focus on eliminating the difference of parameters (or gradients) between the local and global models, which may fail to reflect the model inconsistency due to the complex structure of the machine learning model and the Euclidean space's limitation in meaningful geometric representations. In this paper, we propose FedMRUR by adopting the manifold model fusion scheme and a new global optimizer to alleviate the negative impacts. Concretely, FedMRUR adopts a hyperbolic graph manifold regularizer enforcing the representations of the data in the local and global models are close to each other in a low-dimensional subspace. Because the machine learning model has the graph structure, the distance in hyperbolic space can reflect the model bias better than the Euclidean distance. In this way, FedMRUR exploits the manifold structures of the representations to significantly reduce the model inconsistency. FedMRUR also aggregates the client updates norms as the global update norm, which can appropriately enlarge each client's contribution to the global update, thereby mitigating the norm reduction introduced by the near-orthogonality of client updates. Furthermore, we theoretically prove that our algorithm can achieve a linear speedup property for non-convex setting under partial client participation.Experiments demonstrate that FedMRUR can achieve a new state-of-the-art (SOTA) accuracy with less communication

Joint Power Control and Data Size Selection for Over-the-Air Computation Aided Federated Learning

Federated learning (FL) has emerged as an appealing machine learning approach to deal with massive raw data generated at multiple mobile devices, {which needs to aggregate the training model parameter of every mobile device at one base station (BS) iteratively}. For parameter aggregating in FL, over-the-air computation is a spectrum-efficient solution, which allows all mobile devices to transmit their parameter-mapped signals concurrently to a BS. Due to heterogeneous channel fading and noise, there exists difference between the BS's received signal and its desired signal, measured as the mean-squared error (MSE). To minimize the MSE, we propose to jointly optimize the signal amplification factors at the BS and the mobile devices as well as the data size (the number of data samples involved in local training) at every mobile device. The formulated problem is challenging to solve due to its non-convexity. To find the optimal solution, with some simplification on cost function and variable replacement, which still preserves equivalence, we transform the changed problem to be a bi-level problem equivalently. For the lower-level problem, optimal solution is found by enumerating every candidate solution from the Karush-Kuhn-Tucker (KKT) condition. For the upper-level problem, the optimal solution is found by exploring its piecewise convexity. Numerical results show that our proposed method can greatly reduce the MSE and can help to improve the training performance of FL compared with benchmark methods

Competitive information propagation considering local-global prevalence on multi-layer interconnected networks

The popularity of online social networks (OSNs) promotes the co-propagation of multiple types of information. And there exist inevitably competitive interactions between these information, which will significantly affect the spreading trend of each information. Besides, the coupled topology of multi-layer interconnects exhibited in OSNs will also increase the research complexity of information propagation dynamics. To effectively address these challenges, we propose a novel competitive information propagation model on multi-layer interconnected networks, where the tendency of an individual to become a positive or negative spreader depends on the weighted consideration of local and global prevalence. Then the basic reproduction number is calculated via next-generation matrix method. And under the critical conditions of the basic reproduction number, the asymptotic stability of information-free and information-endemic equilibria is theoretically proven through Lyapunov stability theory. Besides, an optimal control problem involving two heterogeneous controls is formulated, aiming at achieving the best suppression performance of negative information with the minimum control cost. According to Cesari theorem and Pontryagin minimum principle, the existence and analytical formulation of optimal solutions are derived. Extensive numerical experiments are conducted to prove the correctness of our theoretical results, and evaluate the effectiveness of our proposed control strategies. This study can provide useful insights into the modeling and control of multiple information propagation considering multi-layer network topology and individual adaptive behavior

Reduction in the duration of postoperative fever following NUSS surgery during the COVID-19 pandemic

Abstract Background Our study aimed to compare the prevalence of postoperative fever during the COVID-19 pandemic period with that of the preceding non-pandemic period. Methods A retrospective analysis was conducted on patients with pectus excavatum (PE) undergoing minimally invasive repair (also called NUSS procedure) at Nanjing Children’s Hospital from January 1, 2017 to March 1, 2019 (Group 2019), and from January 1, 2020 to March 1, 2021 (Group 2021). Data from a total of 284 patients, consisting of 200 (70.4%) males and 84 (29.6%) females with an average age of 9.73 ± 3.41 (range, 4 to 17) years, were collected. The presence of post-operative fever (defined as a forehead temperature of 37.5℃ or above within 72 h post-surgery), as well as the time of operation, duration of postoperative mechanical ventilator and urinary catheter use, and length of hospitalization were all assessed in admitted patients from Group 2019 (n = 144) and Group 2021 (n = 140). Postoperative white blood cell (WBC), C-reactive protein (CRP) levels, and prevalence of postoperative complications (i.e., pneumothorax, pulmonary atelectasis, pneumonia, wound infection, and dehiscence) were also determined. Result Our results showed a statistically significant decrease in the incidence of postoperative fever within 24 to 72 h of surgery in patients admitted from Group 2019 as compared to Group 2021 (p  0.05). The average hospitalization length of Group 2021 was significantly shorter than Group 2019 (12.49 ± 2.57 vs. 11.85 ± 2.19 days, p  0.05). Conclusion The prevalence of postoperative fever within 72 h of surgery and the length of hospital stay for patients with PE undergoing NUSS surgery were both decreased in Group 2021. We propose that the above phenomenon may be related to increased used of personal protection equipment (such as surgical masks and filtering facepiece respirators (FFRs)) by physicians, nurses, and the patients themselves

Bioinformatic analysis identifies GPR91 as a potential key gene in brain injury after deep hypothermic low flow

Purpose: Explore the transcription change of brain ischemia and reperfusion injury after deep hypothermic low flow. Method: The data from PRJNA739516 and GSE104036 were obtained for the differentially expressed genes identification, functional enrichment analysis, gene set enrichment analysis, protein-protein interaction construction and hub gene identification. Oxygen and glucose deprivation model was set to validate the hub gene and explore the detailed brain injury mechanism. Result: Interleukin, immunological response, NF-κB signaling pathway, G protein-coupled receptor signaling pathway and NLRP inflammatory are functional pathway were enriched in differentially expressed genes analysis. Sucnr1, Casr, Cxcr4, C5ar1, Tas2r41, Tas2r60 and Hcar2 were identified and verified in the OGD model. Knocking down GPR91 reduces the inflammatory response after OGD and GPR91 may be involved in the inflammatory pre-reaction through the synergistic activation of NF-κB, NLRP3, and IL-1β respectively. Conclusion: Our study found that Interleukin, immunological response, NF-κB signaling pathway, G protein-coupled receptor signaling pathway and NLRP inflammatory are all associated with brain ischemia and reperfusion injury after deep hypothermic low flow and GPR91 can activate NF-κB/NLRP3 pathway and trigger the release of IL-1β in this progress

Evaluation and Correlation Analysis of Soil Nutrients and Chemical Constituents of Tobacco Leaves in Meizhou Tobacco Production Area of Guangdong

【Objective】The study aims to explore the traits and their correlation of soil and chemical constituents of tobacco leaves in Meizhou tobacco production area of Guangdong, and explore the key soil factors affecting the chemical quality of tobacco leaves.【Method】48 tobacco-planting soil and the corresponding middle and upper flue-cured tobacco leaves were collected to analysis the traits of soil and chemical constituents of tobacco leaves. Stepwise regression analysis was used to explore the key soil factors influencing the chemical constituents of tobacco leaves.【Result】In the tobacco-growing soil of Meizhou tobacco production area of Guangdong, the proportion of low soil pH value accounts for 64.58%, the content of organic matter (SOM) and water-soluble organic carbon (WSOC) are rich, the appropriate proportion of total nitrogen (TN) content accounts for 68.75%, the rich and extremely rich proportion of total phosphorus (TP) content accounts for 70.83%, the deficient and extremely deficient proportion of total potassium (TK) content accounts for 68.75%, and the rich and extremely rich proportion of available nitrogen (AN), available phosphorus (AP) and available potassium (AK) account for 97.92%, 91.76% and 79.17%, respectively. The content of total sugar, starch and sugar alkali ratio in the middle tobacco leaves were higher, accounting for 91.67%, 100% and 66.67% respectively; the content of total sugar, reducing sugar and starch in the upper tobacco leaves were higher, accounting for 87.50%, 75.00%, 100% and 56.25% respectively; the content of nicotine was lower, accounting for 47.92%. The results of principal component analysis showed that sugar alkali ratio, total nitrogen, nicotine, total sugar and reducing sugar content were the main indicators for evaluating the chemical constituents of middle tobacco leaves, and reducing sugar, sugar alkali ratio, total sugar and nicotine were the main indicators for evaluating the chemical constituents of upper tobacco leaves. Comprehensive chemical quality of middle tobacco leaves was positively correlated with soil AK content, nicotine content in middle tobacco leaves is negatively correlated with soil AK content, the contents of total sugar, reducing sugar and starch in middle tobacco leaves and starch in upper tobacco leaves are negatively correlated with the content of soil WSOC, the content of reducing sugar in upper tobacco leaves is positively correlated with soil AP; the comprehensive chemical quality of upper tobacco leaves was negatively correlated with soil AP content, and positively correlated with soil AK content and pH value.【Conclusion】Improving the soil pH value, AK and WSOC content and reducing the AP content in soil is beneficial to improve the quality of Meizhou tobacco leaves