Measuring Policy Distance for Multi-Agent Reinforcement Learning

Diversity plays a crucial role in improving the performance of multi-agent reinforcement learning (MARL). Currently, many diversity-based methods have been developed to overcome the drawbacks of excessive parameter sharing in traditional MARL. However, there remains a lack of a general metric to quantify policy differences among agents. Such a metric would not only facilitate the evaluation of the diversity evolution in multi-agent systems, but also provide guidance for the design of diversity-based MARL algorithms. In this paper, we propose the multi-agent policy distance (MAPD), a general tool for measuring policy differences in MARL. By learning the conditional representations of agents' decisions, MAPD can computes the policy distance between any pair of agents. Furthermore, we extend MAPD to a customizable version, which can quantify differences among agent policies on specified aspects. Based on the online deployment of MAPD, we design a multi-agent dynamic parameter sharing (MADPS) algorithm as an example of the MAPD's applications. Extensive experiments demonstrate that our method is effective in measuring differences in agent policies and specific behavioral tendencies. Moreover, in comparison to other methods of parameter sharing, MADPS exhibits superior performance.Comment: 9 pages, 6 figure

Learning Heterogeneous Agent Cooperation via Multiagent League Training

Many multiagent systems in the real world include multiple types of agents with different abilities and functionality. Such heterogeneous multiagent systems have significant practical advantages. However, they also come with challenges compared with homogeneous systems for multiagent reinforcement learning, such as the non-stationary problem and the policy version iteration issue. This work proposes a general-purpose reinforcement learning algorithm named as Heterogeneous League Training (HLT) to address heterogeneous multiagent problems. HLT keeps track of a pool of policies that agents have explored during training, gathering a league of heterogeneous policies to facilitate future policy optimization. Moreover, a hyper-network is introduced to increase the diversity of agent behaviors when collaborating with teammates having different levels of cooperation skills. We use heterogeneous benchmark tasks to demonstrate that (1) HLT promotes the success rate in cooperative heterogeneous tasks; (2) HLT is an effective approach to solving the policy version iteration problem; (3) HLT provides a practical way to assess the difficulty of learning each role in a heterogeneous team

Incommensurate itinerant antiferromagnetic excitations and spin resonance in the FeTe0.6_{0.6}Se0.4_{0.4} superconductor

We report on inelastic neutron scattering measurements that find incommensurate itinerant like magnetic excitations in the normal state of superconducting FeTe$_{0.6}$Se$_{0.4}$ (\Tc=14K) at wave-vector $\mathbf{Q}_{inc}=(1/2\pm\epsilon,1/2\mp\epsilon)$ with $\epsilon$=0.09(1). In the superconducting state only the lower energy part of the spectrum shows significant changes by the formation of a gap and a magnetic resonance that follows the dispersion of the normal state excitations. We use a four band model to describe the Fermi surface topology of iron-based superconductors with the extended $s(\pm)$ symmetry and find that it qualitatively captures the salient features of these data.Comment: 7 pages and 5 figure

Qiu, R. et al. A Unified Multi-Functional Dynamic Spectrum Access Framework: Tutorial, Theory and Multi-GHz Wideband Testbed. Sensors 2009, 9, 6530–6603

We found that the affiliation of author Vasu Chakravarthy was incorrect in our paper published in Sensors recently

Vortex-Induced Vibration of a Marine Riser: Numerical Simulation and Mechanism Understanding

Marine riser is a key equipment connecting a floating platform and a seabed wellhead. Vortex-induced vibration (VIV) is the main cause of the fatigue damage of the riser. The prediction of marine riser VIV is very difficult because of its strong non-linearity, instability and uncertainty. In recent years, many numerical models of VIV of marine riser have been developed to explore the mechanism of marine riser VIV, providing scientific theoretical basis and practical engineering methods for vibration control and engineering design of marine riser. Combined with the authors’ own recent research, this chapter discusses the research progress on marine riser VIV in the ocean engineering, including phenomenon mechanism analysis and different numerical research methods

Fracture mechanism of air percussive rotary bit matrix based on impact stress wave theory

An air percussive rotary bit is a key component of air percussive rotary drilling technology, and its fracture failure seriously affects the safe operation and economic efficiency of drilling. This paper presents (1) theoretical analysis of the impact stress wave propagating in the air percussive rotary bit and effect of the stress wave on bit fracture and (2) finite element simulation study based on the stress wave theory which builds a model of the air hammer piston, drill and rock, defines material parameters, meshes and defines boundary conditions, clarifies propagation characteristics of the impact stress wave, analyzes stress characteristics of the bit matrix under different conditions (same drilling pressure and same piston speed, different drilling pressure and same piston speed and same drilling pressure and different piston speed) and determines the main factors of bit matrix fracture. The correctness of the theoretical analysis was verified with simulation results and fundamental ways of preventing bit fracture failure were proposed to provide a theoretical basis for the structural optimization design of a new bit. The results show that a bit section mutation is the root cause for the shock of the impact wave and the change in nature of the wave during propagation. The tensile wave is the root cause for bit matrix fracture, and a breakage is the most serious at stomatal interchanges. With increasing drilling pressure and piston speed, the rate of increase in the peak stress of the bit matrix increases, leading to early fatigue fracture of the bit matrix. The fracture of the bit matrix can be reduced, and the bit life can be extended by rationally designing the bit sectional structure parameters, ensuring that the bit withstands the effects of the compression wave so as to reduce the formation of a tensile wave, and rationally choosing drilling process parameters (such as drilling pressure and air pressure)

Benchmarking Robustness of Multimodal Image-Text Models under Distribution Shift

Multimodal image-text models have shown remarkable performance in the past few years. However, evaluating robustness against distribution shifts is crucial before adopting them in real-world applications. In this work, we investigate the robustness of 12 popular open-sourced image-text models under common perturbations on five tasks (image-text retrieval, visual reasoning, visual entailment, image captioning, and text-to-image generation). In particular, we propose several new multimodal robustness benchmarks by applying 17 image perturbation and 16 text perturbation techniques on top of existing datasets. We observe that multimodal models are not robust to image and text perturbations, especially to image perturbations. Among the tested perturbation methods, character-level perturbations constitute the most severe distribution shift for text, and zoom blur is the most severe shift for image data. We also introduce two new robustness metrics (\textbf{MMI} for MultiModal Impact score and \textbf{MOR} for Missing Object Rate) for proper evaluations of multimodal models. We hope our extensive study sheds light on new directions for the development of robust multimodal models. More details can be found on the project webpage: \url{https://MMRobustness.github.io}.Comment: Accepted by Journal of Data-centric Machine Learning Research (DMLR) 202

Spin Gap and Resonance at the Nesting Wavevector in Superconducting FeSe0.4Te0.6

Neutron scattering is used to probe magnetic excitations in FeSe_{0.4}Te_{0.6} (T_c=14 K). Low energy spin fluctuations are found with a characteristic wave vector $(0.5,0.5,L)$ that corresponds to Fermi surface nesting and differs from Q_m=(\delta,0,0.5) for magnetic ordering in Fe_{1+y}Te. A spin resonance with \hbar\Omega_0=6.5 meV \approx 5.3 k_BT_c and \hbar\Gamma=1.25 meV develops in the superconducting state from a normal state continuum. We show that the resonance is consistent with a bound state associated with s+/- superconductivity and imperfect quasi-2D Fermi surface nesting.Comment: 4 pages, 4 figures, Submitted to Phys. Rev. Let