Generation of Policy-Level Explanations for Reinforcement Learning

Though reinforcement learning has greatly benefited from the incorporation of neural networks, the inability to verify the correctness of such systems limits their use. Current work in explainable deep learning focuses on explaining only a single decision in terms of input features, making it unsuitable for explaining a sequence of decisions. To address this need, we introduce Abstracted Policy Graphs, which are Markov chains of abstract states. This representation concisely summarizes a policy so that individual decisions can be explained in the context of expected future transitions. Additionally, we propose a method to generate these Abstracted Policy Graphs for deterministic policies given a learned value function and a set of observed transitions, potentially off-policy transitions used during training. Since no restrictions are placed on how the value function is generated, our method is compatible with many existing reinforcement learning methods. We prove that the worst-case time complexity of our method is quadratic in the number of features and linear in the number of provided transitions, $O(|F|^2 |tr\_samples|)$. By applying our method to a family of domains, we show that our method scales well in practice and produces Abstracted Policy Graphs which reliably capture relationships within these domains.Comment: Accepted to Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (2019

From liquid to solid bonding in cohesive granular media

We study the transition of a granular packing from liquid to solid bonding in the course of drying. The particles are initially wetted by a liquid brine and the cohesion of the packing is ensured by capillary forces, but the crystallization of the solute transforms the liquid bonds into partially cemented bonds. This transition is evidenced experimentally by measuring the compressive strength of the samples at regular intervals of times. Our experimental data reveal three regimes: 1) Up to a critical degree of saturation, no solid bonds are formed and the cohesion remains practically constant; 2) The onset of cementation occurs at the surface and a front spreads towards the center of the sample with a nonlinear increase of the cohesion; 3) All bonds are partially cemented when the cementation front reaches the center of the sample, but the cohesion increases rapidly due to the consolidation of cemented bonds. We introduce a model based on a parametric cohesion law at the bonds and a bond crystallization parameter. This model predicts correctly the phase transition and the relation between microscopic and macroscopic cohesion.Comment: 20

On the effect of the dynamic contact angle of a vapor embryo interface trapped in a nucleation site

This paper was presented at the 4th Micro and Nano Flows Conference (MNF2014), which was held at University College, London, UK. The conference was organised by Brunel University and supported by the Italian Union of Thermofluiddynamics, IPEM, the Process Intensification Network, the Institution of Mechanical Engineers, the Heat Transfer Society, HEXAG - the Heat Exchange Action Group, and the Energy Institute, ASME Press, LCN London Centre for Nanotechnology, UCL University College London, UCL Engineering, the International NanoScience Community, www.nanopaprika.eu.The effect of boiling and cavitation phenomena on nucleation was first experimentally studied. Results highlight the fact that the "classical" theory of nucleation cannot describe such a configuration. New theoretical approaches were proposed in order to describe the dynamic effects which occur when the liquid pressure oscillates over time and when a heat flux imposed to the system. It then appears that the dynamic and the hysteresis of the contact angle may play a significant role in nucleation by simultaneous boiling and cavitation effects

FROM 3D IMAGING OF STRUCTURES TO DIFFUSIVE PROPERTIES OF ANISOTROPIC CELLULAR MATERIALS

International audienceThis paper deals with diffusive properties phenomena in metallic foams. We have developed a 3D morphological tool to extract geometrical characteristics of the media from X-ray images. The anisotropy of the geometry of each phase is observed and the relationship between microstructure and effective properties is analyzed. We emphasize on geometrical tortuosity determination and impact on con¬ductive transport tensor. The conductive heat transfers are computed on a vertex-edge network to determine directional effective conductivities by solving the energy equa¬tion on this network. We realize a systematic study carried on a wide range of different Nickel foam samples. Finally, we propose a simple model of effective diffusion prop¬erties dependence on tortuosity and porosity

Granular slumping in a fluid : focus on runout distances

We investigate the effect of an ambient fluid on the dynamics of collapse and spread of a granular column simulated by means of a recently developed model which takes into account both fluid forces that act on each grain and contacts between grains. The model couples the contact dynamics method for discrete element modeling of the grains and their interactions with the finite element method for the integration of Navier-Stokes equations in 2D. The coupling is based on the fictitious domain approach in which the fluid domain is extended to that of grains, and the rigid-body motion of the grains is imposed by means of distributed Lagrange multipliers. As in similar numerical and experimental works with dry grains, we focus here on the run-out distances and avalanche durations for different column aspect ratios (height vs width). We consider three options for the surrounding fluid: 1) no fluid, 2) water and 3) a viscous fluid that allows us to perform our simulations in the grain-inertial, fluid-inertial and viscous regimes, respectively. The run-out distance is found to increase as a power law with the aspect ratio of the column, and surprisingly, for a given aspect ratio and packing fraction, it may be similar in the grain-inertial regime and fluid inertial regimes but with considerably longer duration in the latter case. We show that the effect of the fluid in viscous and fluid-inertial regimes is both to reduce the kinetic energy during the collapse and enhance the flow by lubrication during the spread. Hence, the run-out distance in a fluid may be below or equal to that in the absence of fluid due to compensation between those effects