Reinforcement Learning in Sparse-Reward Environments with Hindsight Policy Gradients

A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may also enable higher-level planning based on subgoals. In sparse-reward environments, the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended appears crucial to enabling sample efficient learning. However, reinforcement learning agents have only recently been endowed with such capacity for hindsight. In this letter, we demonstrate how hindsight can be introduced to policy gradient methods, generalizing this idea to a broad class of successful algorithms. Our experiments on a diverse selection of sparse-reward environments show that hindsight leads to a remarkable increase in sample efficiency

Hindsight policy gradients

A reinforcement learning agent that needs to pursue different goals across episodes requires a goal-conditional policy. In addition to their potential to generalize desirable behavior to unseen goals, such policies may also enable higher-level planning based on subgoals. In sparse-reward environments, the capacity to exploit information about the degree to which an arbitrary goal has been achieved while another goal was intended appears crucial to enable sample efficient learning. However, reinforcement learning agents have only recently been endowed with such capacity for hindsight. In this paper, we demonstrate how hindsight can be introduced to policy gradient methods, generalizing this idea to a broad class of successful algorithms. Our experiments on a diverse selection of sparse-reward environments show that hindsight leads to a remarkable increase in sample efficiency.Comment: Accepted to ICLR 201

Spontaneous crystal coalescence enables highly efficient perovskite solar cells

Perovskite solar cells have recently reached staggering efficiencies, through efforts focused on reducing grain boundaries, by enlarging the size of the crystalline domains that constitute the perovskite films. Here, we demonstrate that smaller crystallites within perovskite films spontaneously coalesce into larger ones, even when complete devices are stored in the dark at room temperature. We show that crystal coalescence greatly improves the performance of state of the art perovskite solar cells. Our results reveal the dynamic nature of the morphology of perovskite films and highlight the crucial role that coalescence plays in producing highly efficient device

A perspective on using experiment and theory to identify design principles in dye-sensitized solar cells

Dye-sensitized solar cells (DSCs) have been the subject of wide-ranging studies for many years because of their potential for large-scale manufacturing using roll-to-roll processing allied to their use of earth abundant raw materials. Two main challenges exist for DSC devices to achieve this goal; uplifting device efficiency from the 12 to 14% currently achieved for laboratory-scale ‘hero’ cells and replacement of the widely-used liquid electrolytes which can limit device lifetimes. To increase device efficiency requires optimized dye injection and regeneration, most likely from multiple dyes while replacement of liquid electrolytes requires solid charge transporters (most likely hole transport materials – HTMs). While theoretical and experimental work have both been widely applied to different aspects of DSC research, these approaches are most effective when working in tandem. In this context, this perspective paper considers the key parameters which influence electron transfer processes in DSC devices using one or more dye molecules and how modelling and experimental approaches can work together to optimize electron injection and dye regeneration. This paper provides a perspective that theory and experiment are best used in tandem to study DSC device

Crystal-Size-Induced Band Gap Tuning in Perovskite Films

A comprehensive picture explaining the effect of the crystal size in metal halide perovskite films on their opto-electronic characteristics is currently lacking. We report that perovskite nanocrystallites exhibit a wider band gap due to concurrent quantum confinement and size dependent structural effects, with the latter being remarkably distinct and attributed to the perturbation from the surface of the nanocrystallites affecting the structure of their core. This phenomenon might assist in the photo-induced charge separation within the perovskite in devices employing mesoporous layers as they restrict the size of nanocrystallites present in them. We demonstrate that the crystal size effect is widely applicable as it is ubiquitous in different compositions and deposition methods employed in the fabrication of state-of-the-art perovskite solar cells. This effect is a convenient and effective way to tune the band gap of perovskites