Distributed Learning over Networks with Graph-Attention-Based Personalization

In conventional distributed learning over a network, multiple agents collaboratively build a common machine learning model. However, due to the underlying non-i.i.d. data distribution among agents, the unified learning model becomes inefficient for each agent to process its locally accessible data. To address this problem, we propose a graph-attention-based personalized training algorithm (GATTA) for distributed deep learning. The GATTA enables each agent to train its local personalized model while exploiting its correlation with neighboring nodes and utilizing their useful information for aggregation. In particular, the personalized model in each agent is composed of a global part and a node-specific part. By treating each agent as one node in a graph and the node-specific parameters as its features, the benefits of the graph attention mechanism can be inherited. Namely, instead of aggregation based on averaging, it learns the specific weights for different neighboring nodes without requiring prior knowledge about the graph structure or the neighboring nodes' data distribution. Furthermore, relying on the weight-learning procedure, we develop a communication-efficient GATTA by skipping the transmission of information with small aggregation weights. Additionally, we theoretically analyze the convergence properties of GATTA for non-convex loss functions. Numerical results validate the excellent performances of the proposed algorithms in terms of convergence and communication cost.Comment: Accepted for publication in IEEE TSP; with supplementary details for the derivation

Integrating Over-the-Air Federated Learning and Non-Orthogonal Multiple Access: What Role can RIS Play?

With the aim of integrating over-the-air federated learning (AirFL) and non-orthogonal multiple access (NOMA) into an on-demand universal framework, this paper proposes a novel reconfigurable intelligent surface (RIS)-aided hybrid network by leveraging the RIS to flexibly adjust the signal processing order of heterogeneous data. The objective of this work is to maximize the achievable hybrid rate by jointly optimizing the transmit power, controlling the receive scalar, and designing the phase shifts. Since the concurrent transmissions of all computation and communication signals are aided by the discrete phase shifts at the RIS, the considered problem (P0) is a challenging mixed integer programming problem. To tackle this intractable issue, we decompose the original problem (P0) into a non-convex problem (P1) and a combinatorial problem (P2), which are characterized by the continuous and discrete variables, respectively. For the transceiver design problem (P1), the power allocation subproblem is first solved by invoking the difference-of-convex programming, and then the receive control subproblem is addressed by using the successive convex approximation, where the closed-form expressions of simplified cases are derived to obtain deep insights. For the reflection design problem (P2), the relaxation-then-quantization method is adopted to find a suboptimal solution for striking a trade-off between complexity and performance. Afterwards, an alternating optimization algorithm is developed to solve the non-linear and non-convex problem (P0) iteratively. Finally, simulation results reveal that 1) the proposed RIS-aided hybrid network can support the on-demand communication and computation efficiently, 2) the performance gains can be improved by properly selecting the location of the RIS, and 3) the designed algorithms are also applicable to conventional networks with only AirFL or NOMA users

STAR-RIS Enabled Heterogeneous Networks: Ubiquitous NOMA Communication and Pervasive Federated Learning

This paper integrates non-orthogonal multiple access (NOMA) and over-the-air federated learning (AirFL) into a unified framework using a simultaneous transmitting and reflecting reconfigurable intelligent surface (STAR-RIS). The STAR-RIS plays an important role in adjusting the decoding order of hybrid users for efficient interference mitigation and omni-directional coverage extension. To capture the impact of non-ideal wireless channels on AirFL, a closed-form expression for the optimality gap (a.k.a. convergence upper bound) between the actual loss and the optimal loss is derived. This analysis reveals that the learning performance is significantly affected by active and passive beamforming schemes as well as wireless noise. Furthermore, when the learning rate diminishes as the training proceeds, the optimality gap is explicitly characterized to converge with a linear rate. To accelerate convergence while satisfying QoS requirements, a mixed-integer non-linear programming (MINLP) problem is formulated by jointly designing the transmit power at users and the configuration mode of STAR-RIS. Next, a trust region-based successive convex approximation method and a penalty-based semidefinite relaxation approach is proposed to handle the decoupled non-convex subproblems iteratively. An alternating optimization algorithm is then developed to find a suboptimal solution for the original MINLP problem. Extensive simulation results show that i) the proposed framework can efficiently support NOMA and AirFL users via concurrent uplink communications, ii) our algorithms can achieve a faster convergence rate on the IID and non-IID settings as compared to baselines, and iii) both the spectrum efficiency and learning performance can be significantly improved with the aid of the well-tuned STAR-RIS.Comment: 16 pages, 8 figure

Over-the-Air Split Machine Learning in Wireless MIMO Networks

In split machine learning (ML), different partitions of a neural network (NN) are executed by different computing nodes, requiring a large amount of communication cost. To ease communication burden, over-the-air computation (OAC) can efficiently implement all or part of the computation at the same time of communication. Based on the proposed system, the system implementation over wireless network is introduced and we provide the problem formulation. In particular, we show that the inter-layer connection in a NN of any size can be mathematically decomposed into a set of linear precoding and combining transformations over MIMO channels. Therefore, the precoding matrix at the transmitter and the combining matrix at the receiver of each MIMO link, as well as the channel matrix itself, can jointly serve as a fully connected layer of the NN. The generalization of the proposed scheme to the conventional NNs is also introduced. Finally, we extend the proposed scheme to the widely used convolutional neural networks and demonstrate its effectiveness under both the static and quasi-static memory channel conditions with comprehensive simulations. In such a split ML system, the precoding and combining matrices are regarded as trainable parameters, while MIMO channel matrix is regarded as unknown (implicit) parameters.Comment: 15 pages, 13 figures, journal pape

A comparative study on microstructure and properties of traditional laser cladding and high-speed laser cladding of Ni45 alloy coatings

High-speed laser cladding technology can significantly improve the efficiency of coating preparation and effectively widen the application range of laser cladding. In this study, the Ni45 powders were deposited on steel substrate by traditional low speed laser cladding and high-speed laser cladding process, respectively. The cladding efficiency, surface forming, cross-sectional microstructure, microhardness, wear and corrosion resistance properties of the traditional and high-speed laser cladded Ni45 alloy coatings were compared. It can be seen that the thickness of the high-speed laser cladding coating was much thinner than that of the traditional laser cladding coating. Compared with traditional laser cladding, high-speed laser cladding could achieve a cladding speed of 76.86 m/min and a cladding efficiency of 156.79 cm2/min. The microstructure of the two kinds of coatings shows the same growth law, but the microstructure in high-speed laser cladding was smaller and denser, and the columnar crystal interval was narrower, only about 6 μm. It is found that the cooling rate of the traditional laser cladding coating was smaller than that of the high-speed laser cladding, and as the cladding speed increased, the cooling rate became higher and higher. The cross-section microhardness of the traditional laser cladding coating was relatively uniform of 337 HV0.2, while the microhardness of high-speed laser cladding surface increased to about 543 HV0.2. In addition, the wear and corrosion resistance of high-speed laser cladded coatings were better than that of traditional laser cladded coatings. As the cladding speed increased, the wear and corrosion resistance of the cladded coatings became better