Lyapunov-Based Reinforcement Learning for Decentralized Multi-Agent Control

Decentralized multi-agent control has broad applications, ranging from multi-robot cooperation to distributed sensor networks. In decentralized multi-agent control, systems are complex with unknown or highly uncertain dynamics, where traditional model-based control methods can hardly be applied. Compared with model-based control in control theory, deep reinforcement learning (DRL) is promising to learn the controller/policy from data without the knowing system dynamics. However, to directly apply DRL to decentralized multi-agent control is challenging, as interactions among agents make the learning environment non-stationary. More importantly, the existing multi-agent reinforcement learning (MARL) algorithms cannot ensure the closed-loop stability of a multi-agent system from a control-theoretic perspective, so the learned control polices are highly possible to generate abnormal or dangerous behaviors in real applications. Hence, without stability guarantee, the application of the existing MARL algorithms to real multi-agent systems is of great concern, e.g., UAVs, robots, and power systems, etc. In this paper, we aim to propose a new MARL algorithm for decentralized multi-agent control with a stability guarantee. The new MARL algorithm, termed as a multi-agent soft-actor critic (MASAC), is proposed under the well-known framework of "centralized-training-with-decentralized-execution". The closed-loop stability is guaranteed by the introduction of a stability constraint during the policy improvement in our MASAC algorithm. The stability constraint is designed based on Lyapunov's method in control theory. To demonstrate the effectiveness, we present a multi-agent navigation example to show the efficiency of the proposed MASAC algorithm.Comment: Accepted to The 2nd International Conference on Distributed Artificial Intelligenc

Singular Cucker-Smale Dynamics

The existing state of the art for singular models of flocking is overviewed, starting from microscopic model of Cucker and Smale with singular communication weight, through its mesoscopic mean-filed limit, up to the corresponding macroscopic regime. For the microscopic Cucker-Smale (CS) model, the collision-avoidance phenomenon is discussed, also in the presence of bonding forces and the decentralized control. For the kinetic mean-field model, the existence of global-in-time measure-valued solutions, with a special emphasis on a weak atomic uniqueness of solutions is sketched. Ultimately, for the macroscopic singular model, the summary of the existence results for the Euler-type alignment system is provided, including existence of strong solutions on one-dimensional torus, and the extension of this result to higher dimensions upon restriction on the smallness of initial data. Additionally, the pressureless Navier-Stokes-type system corresponding to particular choice of alignment kernel is presented, and compared - analytically and numerically - to the porous medium equation

Measurement of the 1s-2s energy interval in muonium

The 1s-2s interval has been measured in the muonium ({$\mu^+e^-$}) atom by Doppler-free two-photon laser spectroscopy. The frequency separation of the states was determined to be 2 455 528 941.0(9.8)~MHz in good agreement with quantum electrodynamics. The muon-electron mass ratio can be extracted and is found to be 206.768 38(17). The result may be interpreted as measurement of the muon-electron charge ratio as $-1- 1.1(2.1)\cdot 10^{-9}$

Remotely induced magnetism in a normal metal using a superconducting spin-valve

Superconducting spintronics has emerged in the past decade as a promising new field that seeks to open a new dimension for nanoelectronics by utilizing the internal spin structure of the superconducting Cooper pair as a new degree of freedom1, 2. Its basic building blocks are spin-triplet Cooper pairs with equally aligned spins, which are promoted by proximity of a conventional superconductor to a ferromagnetic material with inhomogeneous macroscopic magnetization3. Using low-energy muon spin-rotation experiments we find an unanticipated effect, in contradiction with the existing theoretical models of superconductivity and ferromagnetism: the appearance of a magnetization in a thin layer of a non-magnetic metal (gold), separated from a ferromagnetic double layer by a 50-nm-thick superconducting layer of Nb. The effect can be controlled either by temperature or by using a magnetic field to control the state of the remote ferromagnetic elements, and may act as a basic building block for a new generation of quantum interference devices based on the spin of a Cooper pair