Learning to Communicate with Deep Multi-Agent Reinforcement Learning

We consider the problem of multiple agents sensing and acting in environments with the goal of maximising their shared utility. In these environments, agents must learn communication protocols in order to share information that is needed to solve the tasks. By embracing deep neural networks, we are able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability. We propose two approaches for learning in these domains: Reinforced Inter-Agent Learning (RIAL) and Differentiable Inter-Agent Learning (DIAL). The former uses deep Q-learning, while the latter exploits the fact that, during learning, agents can backpropagate error derivatives through (noisy) communication channels. Hence, this approach uses centralised learning but decentralised execution. Our experiments introduce new environments for studying the learning of communication protocols and present a set of engineering innovations that are essential for success in these domains

Gravitational Model of High Energy Particles in a Collimated Jet

Observations suggest that relativistic particles play a fundamental role in the dynamics of jets emerging from active galactic nuclei as well as in their interaction with the intracluster medium. However, no general consensus exists concerning the acceleration mechanism of those high energy particles. A gravitational acceleration mechanism is here proposed, in which particles leaving precise regions within the ergosphere of a rotating supermassive black hole produce a highly collimated flow. These particles follow unbound geodesics which are asymptotically parallel to the spin axis of the black hole and are characterized by the energy $E$, the Carter constant ${\cal Q}$ and zero angular momentum of the component $L_z$. If environmental effects are neglected, the present model predicts at distances of about 140 kpc from the ergosphere the presence of electrons with energies around 9.4 GeV. The present mechanism can also accelerate protons up to the highest energies observed in cosmic rays by the present experiments.Comment: 27 pages and 5 figures. Accepted for publication in Astrophysical Journal. arXiv admin note: text overlap with arXiv:1011.654

The compound Poisson limit ruling periodic extreme behaviour of non-uniformly hyperbolic dynamics

We prove that the distributional limit of the normalised number of returns to small neighbourhoods of periodic points of non-uniformly hyperbolic dynamical systems is compound Poisson. The returns to small balls around a fixed point in the phase space correspond to the occurrence of rare events, or exceedances of high thresholds, so that there is a connection between the laws of Return Times Statistics and Extreme Value Laws. The fact that the fixed point in the phase space is a repelling periodic point implies that there is a tendency for the exceedances to appear in clusters whose average sizes is given by the Extremal Index, which depends on the expansion of the system at the periodic point. We recall that for generic points, the exceedances, in the limit, are singular and occur at Poisson times. However, around periodic points, the picture is different: the respective point processes of exceedances converge to a compound Poisson process, so instead of single exceedances, we have entire clusters of exceedances occurring at Poisson times with a geometric distribution ruling its multiplicity. The systems to which our results apply include: general piecewise expanding maps of the interval (Rychlik maps), maps with indifferent fixed points (Manneville-Pomeau maps) and Benedicks-Carleson quadratic maps.Comment: To appear in Communications in Mathematical Physic

Phase Transition and Monopoles Densities in a Nearest Neighbors Two-Dimensional Spin Ice Model

In this work, we show that, due to the alternating orientation of the spins in the ground state of the artificial square spin ice, the influence of a set of spins at a certain distance of a reference spin decreases faster than the expected result for the long range dipolar interaction, justifying the use of the nearest neighbor two dimensional square spin ice model as an effective model. Using an extension of the model presented in ref. [Scientific Reports 5, 15875 (2015)], considering the influence of the eight nearest neighbors of each spin on the lattice, we analyze the thermodynamics of the model and study the monopoles and string densities dependence as a function of the temperature.Comment: 11 pages, 8 figure

Coalescence Rate of Supermassive Black Hole Binaries Derived from Cosmological Simulations: Detection Rates for LISA and ET

The coalescence history of massive black holes has been derived from cosmological simulations, in which the evolution of those objects and that of the host galaxies are followed in a consistent way. The present study indicates that supermassive black holes having masses greater than $\sim 10^{9} M_{\odot}$ underwent up to 500 merger events along their history. The derived coalescence rate per comoving volume and per mass interval permitted to obtain an estimate of the expected detection rate distribution of gravitational wave signals ("ring-down") along frequencies accessible by the planned interferometers either in space (LISA) or in the ground (Einstein). For LISA, in its original configuration, a total detection rate of about $15 yr^{-1}$ is predicted for events having a signal-to-noise ratio equal to 10, expected to occur mainly in the frequency range $4-9 mHz$. For the Einstein gravitational wave telescope, one event each 14 months down to one event each 4 years is expected with a signal-to-noise ratio of 5, occurring mainly in the frequency interval $10-20 Hz$. The detection of these gravitational signals and their distribution in frequency would be in the future an important tool able to discriminate among different scenarios explaining the origin of supermassive black holes.Comment: 18 pages, 7 figures, to appear in the IJMP