Distral: Robust Multitask Reinforcement Learning

Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tasks. In practice, however, this is not usually observed, because gradients from different tasks can interfere negatively, making learning unstable and sometimes even less data efficient. Another issue is the different reward schemes between tasks, which can easily lead to one task dominating the learning of a shared model. We propose a new approach for joint training of multiple tasks, which we refer to as Distral (Distill & transfer learning). Instead of sharing parameters between the different workers, we propose to share a "distilled" policy that captures common behaviour across tasks. Each worker is trained to solve its own task while constrained to stay close to the shared policy, while the shared policy is trained by distillation to be the centroid of all task policies. Both aspects of the learning process are derived by optimizing a joint objective function. We show that our approach supports efficient transfer on complex 3D environments, outperforming several related methods. Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning

Charge migration in organic materials: Can propagating charges affect the key physical quantities controlling their motion?

Charge migration is a ubiquitous phenomenon with profound implications throughout many areas of chemistry, physics, biology and materials science. The long-term vision of designing functional materials with tailored molecular scale properties has triggered an increasing quest to identify prototypical systems where truly molecular conduction pathways play a fundamental role. Such pathways can be formed due to the molecular organization of various organic materials and are widely used to discuss electronic properties at the nanometer scale. Here, we present a computational methodology to study charge propagation in organic molecular stacks at nano and sub-nanoscales and exploit this methodology to demonstrate that moving charge carriers strongly affect the values of the physical quantities controlling their motion. The approach is also expected to find broad application in the field of charge migration in soft matter systems.Comment: 18 pages, 6 figures, accepted for publication in the Israel Journal of Chemistr

Sensitivity of night cooling performance to room/system design: surrogate models based on CFD

Night cooling, especially in offices, attracts growing interest. Unfortunately, building designers face considerable problems with the case-specific convective heat transfer by night. The BES programs they use actually need extra input, from either costly experiments or CFD simulations. Alternatively, up-front research on how to engineer best a generic night cooled office – as in this work – can thrust the application of night cooling. A fully automated configuration of data sampling, geometry/grid generation, CFD solving and surrogate modelling, generates several surrogate models. These models relate the convective heat flow in a night cooled landscape office to the ventilation concept, mass distribution, geometry and driving force for convective heat transfer. The results indicate that cases with a thermally massive floor have the highest night cooling performance

The albedo of snow for partially cloudy skies

The input parameters of the model are atmospheric precipitable water, ozone content, turbidity, cloud optical thickness, size and shape of ice crystal of snow and surface pressure. The model outputs spectral and integrated solar flux snow reflectance as a function of solar elevation and fractional cloudcover. The model is illustrated using representative parameters for the Antarctic coastal regions. The albedo for a clear sky depends inversely on the solar elevation. At high elevation the albedo depends primarily upon the grain size; at low elevation this dependence is on grain size and shape. The gradient of the albedo-elevation curve increases as the grains get larger and faceted. The albedo for a dense overcast is a few percent higher than the clear sky albedo at high elevations. A simple relation between the grain size and the overcast albedo is obtained. For a set of grain size and shape, the albedo matrices (the albedo as a function of solar elevation and fractional cloudcover) are tabulated

Development of an Advanced Force Field for Water using Variational Energy Decomposition Analysis

Given the piecewise approach to modeling intermolecular interactions for force fields, they can be difficult to parameterize since they are fit to data like total energies that only indirectly connect to their separable functional forms. Furthermore, by neglecting certain types of molecular interactions such as charge penetration and charge transfer, most classical force fields must rely on, but do not always demonstrate, how cancellation of errors occurs among the remaining molecular interactions accounted for such as exchange repulsion, electrostatics, and polarization. In this work we present the first generation of the (many-body) MB-UCB force field that explicitly accounts for the decomposed molecular interactions commensurate with a variational energy decomposition analysis, including charge transfer, with force field design choices that reduce the computational expense of the MB-UCB potential while remaining accurate. We optimize parameters using only single water molecule and water cluster data up through pentamers, with no fitting to condensed phase data, and we demonstrate that high accuracy is maintained when the force field is subsequently validated against conformational energies of larger water cluster data sets, radial distribution functions of the liquid phase, and the temperature dependence of thermodynamic and transport water properties. We conclude that MB-UCB is comparable in performance to MB-Pol, but is less expensive and more transferable by eliminating the need to represent short-ranged interactions through large parameter fits to high order polynomials