Control Regularization for Reduced Variance Reinforcement Learning

Dealing with high variance is a significant challenge in model-free reinforcement learning (RL). Existing methods are unreliable, exhibiting high variance in performance from run to run using different initializations/seeds. Focusing on problems arising in continuous control, we propose a functional regularization approach to augmenting model-free RL. In particular, we regularize the behavior of the deep policy to be similar to a policy prior, i.e., we regularize in function space. We show that functional regularization yields a bias-variance trade-off, and propose an adaptive tuning strategy to optimize this trade-off. When the policy prior has control-theoretic stability guarantees, we further show that this regularization approximately preserves those stability guarantees throughout learning. We validate our approach empirically on a range of settings, and demonstrate significantly reduced variance, guaranteed dynamic stability, and more efficient learning than deep RL alone.Comment: Appearing in ICML 201

Superquadrics for segmentation and modeling range data

We present a novel approach to reliable and efficient recovery of part-descriptions in terms of superquadric models from range data. We show that superquadrics can directly be recovered from unsegmented data, thus avoiding any presegmentation steps (e.g., in terms of surfaces). The approach is based on the recover-andselect paradigm. We present several experiments on real and synthetic range images, where we demonstrate the stability of the results with respect to viewpoint and noise

Coalescence Rate of Supermassive Black Hole Binaries Derived from Cosmological Simulations: Detection Rates for LISA and ET

The coalescence history of massive black holes has been derived from cosmological simulations, in which the evolution of those objects and that of the host galaxies are followed in a consistent way. The present study indicates that supermassive black holes having masses greater than $\sim 10^{9} M_{\odot}$ underwent up to 500 merger events along their history. The derived coalescence rate per comoving volume and per mass interval permitted to obtain an estimate of the expected detection rate distribution of gravitational wave signals ("ring-down") along frequencies accessible by the planned interferometers either in space (LISA) or in the ground (Einstein). For LISA, in its original configuration, a total detection rate of about $15 yr^{-1}$ is predicted for events having a signal-to-noise ratio equal to 10, expected to occur mainly in the frequency range $4-9 mHz$. For the Einstein gravitational wave telescope, one event each 14 months down to one event each 4 years is expected with a signal-to-noise ratio of 5, occurring mainly in the frequency interval $10-20 Hz$. The detection of these gravitational signals and their distribution in frequency would be in the future an important tool able to discriminate among different scenarios explaining the origin of supermassive black holes.Comment: 18 pages, 7 figures, to appear in the IJMP

Jets in Hadron-Hadron Collisions

In this article, we review some of the complexities of jet algorithms and of the resultant comparisons of data to theory. We review the extensive experience with jet measurements at the Tevatron, the extrapolation of this acquired wisdom to the LHC and the differences between the Tevatron and LHC environments. We also describe a framework (SpartyJet) for the convenient comparison of results using different jet algorithms.Comment: 68 pages, 54 figure

N-Body Simulation of Planetesimal Formation through Gravitational Instability of a Dust Layer in Laminar Gas Disk

We investigate the formation process of planetesimals from the dust layer by the gravitational instability in the gas disk using local $N$-body simulations. The gas is modeled as a background laminar flow. We study the formation process of planetesimals and its dependence on the strength of the gas drag. Our simulation results show that the formation process is divided into three stages qualitatively: the formation of wake-like density structures, the creation of planetesimal seeds, and their collisional growth. The linear analysis of the dissipative gravitational instability shows that the dust layer is secularly unstable although Toomre's $Q$ value is larger than unity. However, in the initial stage, the growth time of the gravitational instability is longer than that of the dust sedimentation and the decrease in the velocity dispersion. Thus, the velocity dispersion decreases and the disk shrinks vertically. As the velocity dispersion becomes sufficiently small, the gravitational instability finally becomes dominant. Then wake-like density structures are formed by the gravitational instability. These structures fragment into planetesimal seeds. The seeds grow rapidly owing to mutual collisions.Comment: 32 pages, 11 figures, accepted for publication in Ap