Learning to Run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments

In the NIPS 2017 Learning to Run challenge, participants were tasked with building a controller for a musculoskeletal model to make it run as fast as possible through an obstacle course. Top participants were invited to describe their algorithms. In this work, we present eight solutions that used deep reinforcement learning approaches, based on algorithms such as Deep Deterministic Policy Gradient, Proximal Policy Optimization, and Trust Region Policy Optimization. Many solutions use similar relaxations and heuristics, such as reward shaping, frame skipping, discretization of the action space, symmetry, and policy blending. However, each of the eight teams implemented different modifications of the known algorithms.Comment: 27 pages, 17 figure

The U-band Galaxy Luminosity Function of Nearby Clusters

Despite the great potential of the U-band galaxy luminosity function (GLF) to constrain the history of star formation in clusters, to clarify the question of variations of the GLF across filter bands, to provide a baseline for comparisons to high-redshift studies of the cluster GLF, and to estimate the contribution of bound systems of galaxies to the extragalactic near-UV background, determinations have so far been hampered by the generally low efficiency of detectors in the U-band and by the difficulty of constructing both deep and wide surveys. In this paper, we present U-band GLFs of three nearby, rich clusters to a limit of M_U=-17.5 (M*_U+2). Our analysis is based on a combination of separate spectroscopic and R-band and U-band photometric surveys. For this purpose, we have developed a new maximum-likelihood algorithm for calculating the luminosity function that is particularly useful for reconstructing the galaxy distribution function in multi-dimensional spaces (e.g., the number of galaxies as a simultaneous function of luminosity in different filter bands, surface brightness, star formation rate, morphology, etc.), because it requires no prior assumptions as to the shape of the distribution function. The composite luminosity function can be described by a Schechter function with characteristic magnitude M*_U=-19.82+/-0.27 and faint end slope alpha_U=-1.09+/-0.18. The total U-band GLF is slightly steeper than the R-band GLF, indicating that cluster galaxies are bluer at fainter magnitudes. Quiescent galaxies dominate the cumulative U-band flux for M_U<-14. The contribution of galaxies in nearby clusters to the U-band extragalactic background is <1% Gyr^-1 for clusters of masses ~3*10^14 to 2*10^15 M_solar.Comment: 44 pages, 11 figures, accepted for publication in Ap

Empirically Derived Suitability Maps to Downscale Aggregated Land Use Data

Understanding mechanisms that drive present land use patterns is essential in order to derive appropriate models of land use change. When static analyses of land use drivers are performed, they rarely explicitly deal with spatial autocorrelation. Most studies are undertaken on autocorrelation-free data samples. By doing this, a great deal of information that is present in the dataset is lost. This paper presents a spatially explicit, cross-sectional, logistic analysis of land use drivers in Belgium. It is shown that purely regressive logistic models can only identify trends or global relationships between socio-economic or physico-climatic drivers and the precise location of each land use type. However, when the goal of a study is to obtain the best model of land use distribution, a purely autoregressive (or neighbourhood-based) model is appropriate. Moreover, it is also concluded that a neighbourhood based only on the 8 surrounding cells leads to the best logistic regression models at this scale of observation. This statement is valid for each land use type studied – i.e. built-up, forests, cropland and grassland.

Multiple Parton Interactions in Hadron Collisions and Diffraction

Hadrons are composite objects made of quarks and gluons, and during a collision one can have several elementary interactions between the constituents. These elementary interactions, using an appropriate theoretical framework, can be related to the total and elastic cross sections. At high c.m. energy it also becomes possible to identify experimentally a high pt subset of the parton interactions and to study their multiplicity distribution. Predictions of the multiple interactions rates are difficult because in principle one needs to have a knowledge of the correlated Parton Distribution Functions that describe the probability to find simultaneously different partons in different elements of phase space. In this work we address this question and suggest a method to describe effectively the fluctuations in the instantaneous configuration of a colliding hadron. This problem is intimately related to the origin of the inelastic diffractive processes. We present a new method to include the diffractive cross section in an eikonal formalism that is equivalent to a multi-channel eikonal. We compare with data and present an extrapolation to higher energy.Comment: 34 pages, 9 figure