Settling the Variance of Multi-Agent Policy Gradients

Policy gradient (PG) methods are popular reinforcement learning (RL) methods where a baseline is often applied to reduce the variance of gradient estimates. In multi-agent RL (MARL), although the PG theorem can be naturally extended, the effectiveness of multi-agent PG (MAPG) methods degrades as the variance of gradient estimates increases rapidly with the number of agents. In this paper, we offer a rigorous analysis of MAPG methods by, firstly, quantifying the contributions of the number of agents and agents’ explorations to the variance of MAPG estimators. Based on this analysis, we derive the optimal baseline (OB) that achieves the minimal variance. In comparison to the OB, we measure the excess variance of existing MARL algorithms such as vanilla MAPG and COMA. Considering using deep neural networks, we also propose a surrogate version of OB, which can be seamlessly plugged into any existing PG methods in MARL. On benchmarks of Multi-Agent MuJoCo and StarCraft challenges, our OB technique effectively stabilises training and improves the performance of multi-agent PPO and COMA algorithms by a significant margin. Code is released at https://github.com/morning9393/Optimal-Baseline-for-Multi-agent-Policy-Gradients

Nurses\u27 Alumnae Association Bulletin - Volume 16 Number 1

Alumnae Notes ANA Biennial Convention Cancer of the Cervix, Uterus and Ovaries Committee Reports Digest of Alumnae Association Meetings Greetings from Miss Childs Greetings from the President Graduation Awards - 1950 Isotopes and the Nurse - Dr. T.P. Eberhard Marriages Necrology New Arrivals Nursing Care in Heart Disease with Pulmonary Infarction Nursing Care of a Mitral Commissurotomy Physical Advances at Jefferson - 1950 Policies of the Private Duty Nurses\u27 Registry Staff Activities, 1950-1951 Students\u27 Corner The Department of Surgical Research - Drs. Templeton and Gibbon White Haven and Barton Memorial Division

Influence of Solvent Temperature and Type on Naphthalene Solubility for Tar Removal in a Dual Fluidized Bed Biomass Gasification Process

Tar condensation is a cause of blockage in downstream application of the gasification process. An oil scrubber is considered as an effective method for tar removal. In this research, the naphthalene solubility in different local Thai oils and water was investigated in a laboratory-scale test-rig. The solubility value was conducted at 30, 50, 70, and 80°C. Biodiesels investigated were rapeseed methyl ester (RME) and two different palm methyl esters (PME 1 and PME 2). Furthermore, vegetable oils including sunflower oil, rice bran oil, crude palm oil, and refined palm oil were examined. The results showed that higher temperature enhanced naphthalene solubility in all types of investigated oils. Biodiesel has the highest value of naphthalene solubility. All scrubbing oils have similar naphthalene solubility trends at the temperature range of 50-80°C in the order of RME > PME 1 > PME 2 > diesel > sunflower oil > refined palm oil > rice bran oil > crude palm oil. Based on these experimental investigations, PME 1 has a naphthalene solubility value similar to RME. Therefore, PME 1 has been selected to be tested as scrubbing solvent in the 1 MWel prototype dual fluidized gasifier located in Nong Bua district, Nakhon Sawan province, Thailand

Recording electrical activity from the brain of behaving octopus

: Octopuses, which are among the most intelligent invertebrates,1,2,3,4 have no skeleton and eight flexible arms whose sensory and motor activities are at once autonomous and coordinated by a complex central nervous system.5,6,7,8 The octopus brain contains a very large number of neurons, organized into numerous distinct lobes, the functions of which have been proposed based largely on the results of lesioning experiments.9,10,11,12,13 In other species, linking brain activity to behavior is done by implanting electrodes and directly correlating electrical activity with observed animal behavior. However, because the octopus lacks any hard structure to which recording equipment can be anchored, and because it uses its eight flexible arms to remove any foreign object attached to the outside of its body, in vivo recording of electrical activity from untethered, behaving octopuses has thus far not been possible. Here, we describe a novel technique for inserting a portable data logger into the octopus and implanting electrodes into the vertical lobe system, such that brain activity can be recorded for up to 12 h from unanesthetized, untethered octopuses and can be synchronized with simultaneous video recordings of behavior. In the brain activity, we identified several distinct patterns that appeared consistently in all animals. While some resemble activity patterns in mammalian neural tissue, others, such as episodes of 2 Hz, large amplitude oscillations, have not been reported. By providing an experimental platform for recording brain activity in behaving octopuses, our study is a critical step toward understanding how the brain controls behavior in these remarkable animals

Psychologizing indexes of societal progress: Accounting for cultural diversity in preferred developmental pathways

Since the Second World War, the dominating paradigm of societal development has focused on economic growth. While economic growth has improved the quality of human life in a variety of ways, we posit that the identification of economic growth as the primary societal goal is culture-blind because preferences for developmental pathways likely vary between societies. We argue that the cultural diversity of developmental goals and the pathways leading to these goals could be reflected in a culturally sensitive approach to assessing societal development. For the vast majority of post-materialistic societies, it is an urgent necessity to prepare culturally sensitive compasses on how to develop next, and to start conceptualizing growth in a more nuanced and culturally responsive way. Furthermore, we propose that cultural sensitivity in measuring societal growth could also be applied to existing development indicators (e.g. the Human Development Index). We call for cultural researchers, in cooperation with development economists and other social scientists, to prepare a new cultural map of developmental goals, and to create and adapt development indexes that are more culturally sensitive. This innovation could ultimately help social planners understand the diverse pathways of development and assess the degree to which societies are progressing in a self-determined and indigenously valued manner.info:eu-repo/semantics/acceptedVersio

Cutting edges at random in large recursive trees

We comment on old and new results related to the destruction of a random recursive tree (RRT), in which its edges are cut one after the other in a uniform random order. In particular, we study the number of steps needed to isolate or disconnect certain distinguished vertices when the size of the tree tends to infinity. New probabilistic explanations are given in terms of the so-called cut-tree and the tree of component sizes, which both encode different aspects of the destruction process. Finally, we establish the connection to Bernoulli bond percolation on large RRT's and present recent results on the cluster sizes in the supercritical regime.Comment: 29 pages, 3 figure

A step-down photophobic response in coral larvae: implications for the light-dependent distribution of the common reef coral, Acropora tenuis

Behavioral responses to environmental factors at the planktonic larval stage can have a crucial influence on habitat selection and therefore adult distributions in many benthic organisms. Reef-building corals show strong patterns of zonation across depth or underwater topography, with different suites of species aggregating in different light environments. One potential mechanism driving this pattern is the response of free-swimming larvae to light. However, there is little experimental support for this hypothesis; in particular, there are few direct and quantitative observations of larval behavior in response to light. Here, we analyzed the swimming behavior of larvae of the common reef coral Acropora tenuis under various light conditions. Larvae exhibited a step-down photophobic response, i.e. a marked decrease in swimming speed, in response to a rapid attenuation (step-down) of light intensity. Observations of larvae under different wavelengths indicated that only the loss of blue light (wavelengths between 400 and 500 nm) produced a significant response. Mathematical simulations of this step-down photophobic response indicate that larvae will aggregate in the lighter areas of two-dimensional large rectangular fields. These results suggest that the step-down photophobic response of coral larvae may play an important role in determining where larval settle on the reef