Efficient-Adam: Communication-Efficient Distributed Adam

Distributed adaptive stochastic gradient methods have been widely used for large-scale nonconvex optimization, such as training deep learning models. However, their communication complexity on finding $\varepsilon$-stationary points has rarely been analyzed in the nonconvex setting. In this work, we present a novel communication-efficient distributed Adam in the parameter-server model for stochastic nonconvex optimization, dubbed {\em Efficient-Adam}. Specifically, we incorporate a two-way quantization scheme into Efficient-Adam to reduce the communication cost between the workers and server. Simultaneously, we adopt a two-way error feedback strategy to reduce the biases caused by the two-way quantization on both the server and workers, respectively. In addition, we establish the iteration complexity for the proposed Efficient-Adam with a class of quantization operators, and further characterize its communication complexity between the server and workers when an $\varepsilon$-stationary point is achieved. Finally, we apply Efficient-Adam to solve a toy stochastic convex optimization problem and train deep learning models on real-world vision and language tasks. Extensive experiments together with a theoretical guarantee justify the merits of Efficient Adam.Comment: IEEE Transactions on Signal Processin

Rethinking SIGN Training: Provable Nonconvex Acceleration without First- and Second-Order Gradient Lipschitz

Sign-based stochastic methods have gained attention due to their ability to achieve robust performance despite using only the sign information for parameter updates. However, the current convergence analysis of sign-based methods relies on the strong assumptions of first-order gradient Lipschitz and second-order gradient Lipschitz, which may not hold in practical tasks like deep neural network training that involve high non-smoothness. In this paper, we revisit sign-based methods and analyze their convergence under more realistic assumptions of first- and second-order smoothness. We first establish the convergence of the sign-based method under weak first-order Lipschitz. Motivated by the weak first-order Lipschitz, we propose a relaxed second-order condition that still allows for nonconvex acceleration in sign-based methods. Based on our theoretical results, we gain insights into the computational advantages of the recently developed LION algorithm. In distributed settings, we prove that this nonconvex acceleration persists with linear speedup in the number of nodes, when utilizing fast communication compression gossip protocols. The novelty of our theoretical results lies in that they are derived under much weaker assumptions, thereby expanding the provable applicability of sign-based algorithms to a wider range of problems

Microwave Absorbing Properties and Mechanism Analysis of Ni–Doped Fe–Based Metallic Microwires

Fe–based metallic microwires possess unique microstructure and size effects, exhibiting favorable mechanical, electrical, and magnetic properties, thus distinguishing them as a possible agents for use as microwave absorbing materials. In this paper, the absorbing properties of Ni–doped Fe–based metallic microwires optimized by orthogonal experiments were investigated, and based on the optimal parameters, the influencing mechanism of the Ni doping amount on the absorbing properties was further analyzed. It was noted that at the frequency f = 8.36 GHz, the maximum reflection loss RL and electromagnetic wave absorption efficiency Aeff can reach −54.89 dB and 99.999%, respectively. Moreover, the Ni doping amount could result in the improved wave-absorbing properties of composites, obtain the corresponding optimal parameters, and even change the position of the maximum absorption peak, which are all of great significance for practical engineering applications

Human–Robot Cooperative Strength Training Based on Robust Admittance Control Strategy

A stroke is a common disease that can easily lead to lower limb motor dysfunction in the elderly. Stroke survivors can effectively train muscle strength through leg flexion and extension training. However, available lower limb rehabilitation robots ignore the knee soft tissue protection of the elderly in training. This paper proposes a human–robot cooperative lower limb active strength training based on a robust admittance control strategy. The stiffness change law of the admittance model is designed based on the biomechanics of knee joints, and it can guide the user to make force correctly and reduce the stress on the joint soft tissue. The controller will adjust the model stiffness in real-time according to the knee joint angle and then indirectly control the exertion force of users. This control strategy not only can avoid excessive compressive force on the joint soft tissue but also can enhance the stimulation of quadriceps femoris muscles. Moreover, a dual input robust control is proposed to improve the tracking performance under the disturbance caused by model uncertainty, interaction force and external noise. Experiments about the controller performance and the training feasibility were conducted with eight stroke survivors. Results show that the designed controller can effectively influence the interaction force; it can reduce the possibility of joint soft tissue injury. The robot also has a good tracking performance under disturbances. This control strategy also can enhance the stimulation of quadriceps femoris muscles, which is proved by measuring the muscle electrical signal and interaction force. Human–robot cooperative strength training is a feasible method for training lower limb muscles with the knee soft tissue protection mechanism

Direct Current Annealing Modulated Ordered Structure to Optimize Tensile Mechanical Properties of Co-Based Amorphous Metallic Microwires

Herein, the ordered structure of Co-based metallic microwires was modulated by direct current-annealing, thereby improving the tensile mechanical properties. Based on the thermophysical parameters of the metallic microwires, the annealing current intensities of 65 mA, 90 mA and 150 mA were determined by the method of numerical calculation. The experimental results indicated that the ordered structure of the metallic microwires was regulated under the action of Joule heating, and with the rising of the annealing current, the ordered structure increased and the distribution tended to be concentrated. The 90 mA current-annealed metallic microwires have favorable tensile mechanical properties and fracture reliability, with the tensile strength and elongation of 4540.10 MPa and 2.99%, respectively, and the fracture threshold is 1910.90 MPa. Both the as-cast and current-annealed metallic microwires were brittle fractures, and the fractures consisted of shear deformation regions and crack extension regions. The improvement of the mechanical properties of metallic microwires is related to the nano-ordered structure and their distribution. Under the condition of 90 mA current annealing, the uniformly distributed nano-ordered structures were formed in the amorphous matrix of the metallic microwires, which can effectively slow down the expansion of the shear bands and reduce the possibility of crack generation. This study provides process reference and theoretical guidance for the application of Co-based metallic microwires in the field of stress sensors

Stress torsional magneto-impedance effect and mechanical properties of Co-based metallic microfibers

GMI characteristics of Co-based metallic microfibers under different stresses are tested by stress torsional magneto-impedance comprehensive platform. The internal stress distribution of microfibers is simulated with ANSYS finite element software and the fracture mechanism is analyzed based on fracture morphology. The experimental results indicate that tensile stress can enhance the GMI effect of metallic microfibers, and the significant [ΔZ/Zmax]max of 293.63% at a frequency of 3 MHz is obtained under the tensile stress of 482 MPa. The simulation results demonstrate that the internal stress concentration occurs at clamping end under tensile stress, and which accumulating on the surface of microfibers after applying torsional strain, thus weakening GMI effect. The fracture morphology will change with external stress, some vein-shaped patterns and molten drops can be seen in the cross-section under tensile stress, and the helical vein-shaped patterns appear with the application of torsional strain. In addition, the circular domain wall energy will be increased due to magnetoelastic anisotropy has been affected by tensile stress, which promotes an increment of circular magnetic permeability, thus improving the GMI effect of microfibers. However, the appearance of helical anisotropy after torsional strain is added will weaken the circular domain wall energy

Fluvial sediment source to sink transfer at the Yellow River Delta: Quantifications, causes, and environmental impacts

Intensified human interventions in river basins and deltas lead to more complexities of environmental changes during the Anthropocene. Changes in river regime especially a dramatic reduction in sediment delivery increase challenges of the morphological and ecological sustainability of river deltas. In evaluating deltaic risks and sustainable solutions, researches are often limited to single geomorphic units of the deltaic system, and investigations of sediment source to sink transfer at river deltas under recent river regimes are often missing. The Yellow River Delta (YRD) presents as a typical megadelta under stressors induced by changing environments. This study utilizes a period of 20-yr high-resolution topography data of the deltaic channel and its subaqueous delta to investigate sediment transport and source to sink process by integrated methods of field measurements and numerical simulations. The results indicate that the deltaic channel has transitioned from net accretion to erosion after the implementation of the Water-Sediment Regulation Scheme (WSRS) in 2002. The active river mouth experienced a slow accretion phase since the river channel diverted to Qing 8 channel, with a reduced vertical deposition rate of 0.15 m/yr, whilst its adjacent Gudong littoral zone had a −0.11 m/yr erosion rate. Under the new fluvial regime, the river-borne suspended sediment tends to transport southwards to the Laizhou Bay, followed by the river-derived sediment transport eastward and northward to the offshore delta. It is clear that with the continued human activities in the region, the YRD is at the potential state of deltaic transition both in the deltaic channel and its subaqueous delta. This transition is believed to be beneficial to the deltaic channel stability, but it could significantly impact on the geomorphic and ecologic sustainability of the entire deltaic system