Learning from Monte Carlo Rollouts with Opponent Models for Playing Tron

This paper describes a novel reinforcement learning system for learning to play the game of Tron. The system combines Q-learning, multi-layer perceptrons, vision grids, opponent modelling, and Monte Carlo rollouts in a novel way. By learning an opponent model, Monte Carlo rollouts can be effectively applied to generate state trajectories for all possible actions from which improved action estimates can be computed. This allows to extend experience replay by making it possible to update the state-action values of all actions in a given game state simultaneously. The results show that the use of experience replay that updates the Q-values of all actions simultaneously strongly outperforms the conventional experience replay that only updates the Q-value of the performed action. The results also show that using short or long rollout horizons during training lead to similar good performances against two fixed opponents

Drug delivery and controlled release from biocompatible metal-organic frameworks using mechanical amorphization

We have used a family of Zr-based metal-organic frameworks (MOFs) with different functionalized (bromo, nitro and amino) and extended linkers for drug delivery. We loaded the materials with the fluorescent model molecule calcein and the anticancer drug α-cyano-4-hydroxycinnamic acid (α-CHC), and consequently performed a mechanical amorphization process to attempt to control the delivery of guest molecules. Our analysis revealed that the loading values of both molecules were higher for the MOFs containing unfunctionalized linkers. Confocal microscopy showed that all the materials were able to penetrate into cells, and the therapeutic effect of α-CHC on HeLa cells was enhanced when loaded (20 wt%) into the MOF with the longest linker. On one hand, calcein release required up to 3 days from the crystalline form for all the materials. On the other hand, the amorphous counterparts containing the bromo and nitro functional groups released only a fraction of the total loaded amount, and in the case of the amino-MOF a slow and progressive release was successfully achieved for 15 days. In the case of the materials loaded with α-CHC, no difference was observed between the crystalline and amorphous form of the materials. These results highlight the necessity of a balance between the pore size of the materials and the size of the guest molecules to accomplish a successful and efficient sustained release using this mechanical ball-milling process. Additionally, the endocytic pathway used by cells to internalize these MOFs may lead to diverse final cellular locations and consequently, different therapeutic effects. Understanding these cellular mechanisms will drive the design of more effective MOFs for drug delivery applications.C.A.O. thanks Becas Chile and the Cambridge Trust for funding. D.F.J. thanks the Royal Society (UK) for funding through a University Research Fellowship. RSF thanks the Royal Society for receipt of a University Research Fellowship and the EPSRC (EP/L004461/1) and The University of Glasgow for funding. A.K.C is grateful to the European Research Council for an Advanced Investigator Award

Scale, history and justice in community wind energy: An empirical review

This is the author accepted manuscript. The final version is available from Elsevier via the DOI in this recordAlthough there is a clear positive link between community wind energy (CWE) projects and social acceptance, there is still empirical and conceptual ambiguity concerning the details of why. To fill this gap, we revisit foundational papers in this field and then, focusing on empirical case studies between 2010 and 2018 (n = 15), trace how recent research has engaged with existing conceptual frameworks. Most empirical researchers verify the importance of the two key dimensions defined by Walker & Devine-Wright [1]: process and outcome, and then relate this to procedural justice and distributive justice. Meanwhile, the core concept of “community” has been deployed, in both practice and research, in so many different and sometimes ambiguous ways that it remains difficult to assert if, and how, community-based renewable energy policy and siting practice produces high levels of local community acceptance. We suggest that parsing out the scale of investment in wind energy projects and the local historical context of energy transitions add clarity to the Walker & Devine-Wright framework as it relates to CWE; providing important conceptual nuance for guiding policy, developer practices and future empirical research.Social Sciences and Humanities Research Council of Canad

How classical is the quantum universe?

We discuss two topics that are usually considered to be exclusively "quantum": the Schroedinger equation, and the uncertainty principle. We show (or rather recall) that the Schroedinger equation can be derived from Hamilton's equations using the metaplectic representation. We also show that the uncertainty principle, stated in the form of the Robertson-Schroedinger-Heisenberg inequalities can be formulated in perfectly classical terms using the topological notion of symplectic capacity

ZnO Nanowires Synthesized by Vapor Phase Transport Deposition on Transparent Oxide Substrates

Zinc oxide nanowires have been synthesized without using metal catalyst seed layers on fluorine-doped tin oxide (FTO) substrates by a modified vapor phase transport deposition process using a double-tube reactor. The unique reactor configuration creates a Zn-rich vapor environment that facilitates formation and growth of zinc oxide nanoparticles and wires (20–80 nm in diameter, up to 6 μm in length, density <40 nm apart) at substrate temperatures down to 300°C. Electron microscopy and other characterization techniques show nanowires with distinct morphologies when grown under different conditions. The effect of reaction parameters including reaction time, temperature, and carrier gas flow rate on the size, morphology, crystalline structure, and density of ZnO nanowires has been investigated. The nanowires grown by this method have a diameter, length, and density appropriate for use in fabricating hybrid polymer/metal oxide nanostructure solar cells. For example, it is preferable to have nanowires no more than 40 nm apart to minimize exciton recombination in polymer solar cells

Root Suberin Forms an Extracellular Barrier That Affects Water Relations and Mineral Nutrition in Arabidopsis

Though central to our understanding of how roots perform their vital function of scavenging water and solutes from the soil, no direct genetic evidence currently exists to support the foundational model that suberin acts to form a chemical barrier limiting the extracellular, or apoplastic, transport of water and solutes in plant roots. Using the newly characterized enhanced suberin1 (esb1) mutant, we established a connection in Arabidopsis thaliana between suberin in the root and both water movement through the plant and solute accumulation in the shoot. Esb1 mutants, characterized by increased root suberin, were found to have reduced day time transpiration rates and increased water-use efficiency during their vegetative growth period. Furthermore, these changes in suberin and water transport were associated with decreases in the accumulation of Ca, Mn, and Zn and increases in the accumulation of Na, S, K, As, Se, and Mo in the shoot. Here, we present direct genetic evidence establishing that suberin in the roots plays a critical role in controlling both water and mineral ion uptake and transport to the leaves. The changes observed in the elemental accumulation in leaves are also interpreted as evidence that a significant component of the radial root transport of Ca, Mn, and Zn occurs in the apoplast

Spatio-Temporal Credit Assignment in Neuronal Population Learning

In learning from trial and error, animals need to relate behavioral decisions to environmental reinforcement even though it may be difficult to assign credit to a particular decision when outcomes are uncertain or subject to delays. When considering the biophysical basis of learning, the credit-assignment problem is compounded because the behavioral decisions themselves result from the spatio-temporal aggregation of many synaptic releases. We present a model of plasticity induction for reinforcement learning in a population of leaky integrate and fire neurons which is based on a cascade of synaptic memory traces. Each synaptic cascade correlates presynaptic input first with postsynaptic events, next with the behavioral decisions and finally with external reinforcement. For operant conditioning, learning succeeds even when reinforcement is delivered with a delay so large that temporal contiguity between decision and pertinent reward is lost due to intervening decisions which are themselves subject to delayed reinforcement. This shows that the model provides a viable mechanism for temporal credit assignment. Further, learning speeds up with increasing population size, so the plasticity cascade simultaneously addresses the spatial problem of assigning credit to synapses in different population neurons. Simulations on other tasks, such as sequential decision making, serve to contrast the performance of the proposed scheme to that of temporal difference-based learning. We argue that, due to their comparative robustness, synaptic plasticity cascades are attractive basic models of reinforcement learning in the brain

Reactive oxygen species initiate luminal but not basal cell death in cultured human mammary alveolar structures: a potential regulator of involution

Post-lactational involution of the mammary gland is initiated within days of weaning. Clearing of cells occurs by apoptosis of the milk-secreting luminal cells in the alveoli and through stromal tissue remodeling to return the gland almost completely to its pre-pregnant state. The pathways that specifically target involution of the luminal cells in the alveoli but not the basal and ductal cells are poorly understood. In this study we show in cultured human mammary alveolar structures that the involution process is initiated by fresh media withdrawal, and is characterized by cellular oxidative stress, expression of activated macrophage marker CD68 and finally complete clearing of the luminal but not basal epithelial layer. This process can be simulated by ectopic addition of reactive oxygen species (ROS) in cultures without media withdrawal. Cells isolated from post-involution alveoli were enriched for the CD49f+ mammary stem cell (MaSC) phenotype and were able to reproduce a complete alveolar structure in subcultures without any significant loss in viability. We propose that the ROS produced by accumulated milk breakdown post-weaning may be the mechanism underlying the selective involution of secretory alveolar luminal cells, and that our culture model represents an useful means to investigate this and other mechanisms further

Values associated with public involvement in health and social care research: a narrative review

addresses: Mood Disorders Centre, Psychology, University of Exeter, Exeter, UK.OnlineOpen articleMuch has been written about public involvement (PI) in health and social care research, but underpinning values are rarely made explicit despite the potential for these to have significant influence on the practice and assessment of PI.Medical Research Council’s Methodology Research Programm