Learning to Play Pac-Xon with Q-Learning and Two Double Q-Learning Variants

Pac-Xon is an arcade video game in which the player tries to fill a level space by conquering blocks while being threatened by enemies. In this paper it is investigated whether a reinforcement learning (RL) agent can successfully learn to play this game. The RL agent consists of a multilayer perceptron (MLP) that uses a feature representation of the game state through input variables and gives Q-values for each possible action as output. For training the agent, the use of Q-learning is compared to two double Q-learning variants, the original algorithm and a novel variant. Furthermore, we have set up an alternative reward function which presents higher rewards towards the end of a level to try to increase the performance of the algorithms. The results show that all algorithms can be used to successfully learn to play Pac-Xon. Furthermore both double Q-learning variants obtain significantly higher performances than Q-learning and the progressive reward function does not yield better results than the regular reward function

Translation controls the expression level of a chimaeric reporter gene

Transcriptional and translational fusions between the reading frame of the β-D-glucuronidase gene (gusA) and the 2′ as well as the 1′ promoter of mannopine synthase (mas), a TR locus of Agrobacterium tumefaciens, were made. The expression of these constructs was studied in the transgenic F1 offspring of independent tobacco transformants at the protein level by assaying for GUS activity and western blot analysis of the GUS protein and at the steady-state mRNA level. In leaves, stems and roots no correlation was found between steady-state levels of GUS mRNA and enzyme activity. In older tissues significantly higher GUS activities were found. This is explained by the stable character of the GUS protein together with an accumulation of protein upon ageing. Three to ten times higher GUS activities were found for in vitro grown plants than for greenhouse-grown plants of the same offspring, despite similar levels of GUS mRNA. Roots from in vitro grown plants display three to ten times higher GUS activities than stems and leaves. In transgenic plants grown in vitro, containing a translational fusion with two AUGs in phase, the initiation of translation in leaf material occurred at both AUGs. Initiation of translation at the first AUG, however, was ten times more frequent. In contrast, initiation in roots from in vitro grown plants occurred exclusively at the second AUG

Chronobiology and chronotherapy of osteoporosis

Physiological circadian (ie, 24-hour) rhythms are critical for bone health. Animal studies have shown that genes involved in the intrinsic molecular clock demonstrate potent circadian expression patterns in bone and that genetic disruption of these clock genes results in a disturbed bone structure and quality. More importantly, circulating markers of bone remodeling show diurnal variation in mice as well as humans, and circadian disruption by, eg, working night shifts is associated with the bone remodeling disorder osteoporosis. In this review, we provide an overview of the current literature on rhythmic bone remodeling and its underlying mechanisms and identify critical knowledge gaps. In addition, we discuss novel (chrono)therapeutic strategies to reduce osteoporosis by utilizing our knowledge on circadian regulation of bone. (c) 2021 The Authors. JBMR Plus published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research.Diabetes mellitus: pathophysiological changes and therap

Thermodynamics of a fast-moving Greenlandic outlet glacier revealed by fiber-optic distributed temperature sensing

Funding: This research was funded by the European Research Council as part of the RESPONDER project under the European Union’s Horizon 2020 research and innovation program (grant 683043). R.L. and T.R.C. were supported by Natural Environment Research Council Doctoral Training Partnership studentships (grant NE/ L002507/1). B.H. was supported by a HEFCW/Aberystwyth University Capital Equipment Grant.Measurements of ice temperature provide crucial constraints on ice viscosity and the thermodynamic processes occurring within a glacier. However, such measurements are presently limited by a small number of relatively coarse-spatial-resolution borehole records, especially for ice sheets. Here, we advance our understanding of glacier thermodynamics with an exceptionally high-vertical-resolution (~0.65 m), distributed-fiber-optic temperature-sensing profile from a 1043-m borehole drilled to the base of Sermeq Kujalleq (Store Glacier), Greenland. We report substantial but isolated strain heating within interglacial-phase ice at 208 to 242 m depth together with strongly heterogeneous ice deformation in glacial-phase ice below 889 m. We also observe a high-strain interface between glacial- and interglacial-phase ice and a 73-m-thick temperate basal layer, interpreted as locally formed and important for the glacier's fast motion. These findings demonstrate notable spatial heterogeneity, both vertically and at the catchment scale, in the conditions facilitating the fast motion of marine-terminating glaciers in Greenland.Publisher PDFPeer reviewe

Time-restricted feeding improves adaptation to chronically alternating light-dark cycles

Disturbance of the circadian clock has been associated with increased risk of cardio-metabolic disorders. Previous studies showed that optimal timing of food intake can improve metabolic health. We hypothesized that time-restricted feeding could be a strategy to minimize long term adverse metabolic health effects of shift work and jetlag. In this study, we exposed female FVB mice to weekly alternating light-dark cycles (i.e. 12 h shifts) combined with ad libitum feeding, dark phase feeding or feeding at a fixed clock time, in the original dark phase. In contrast to our expectations, long-term disturbance of the circadian clock had only modest effects on metabolic parameters. Mice fed at a fixed time showed a delayed adaptation compared to ad libitum fed animals, in terms of the similarity in 24 h rhythm of core body temperature, in weeks when food was only available in the light phase. This was accompanied by increased plasma triglyceride levels and decreased energy expenditure, indicating a less favorable metabolic state. On the other hand, dark phase feeding accelerated adaptation of core body temperature and activity rhythms, however, did not improve the metabolic state of animals compared to ad libitum feeding. Taken together, restricting food intake to the active dark phase enhanced adaptation to shifts in the light-dark schedule, without significantly affecting metabolic parameters

Plant-inducible virulence promoter of the Agrobacterium tumefaciens Ti plasmid

Agrobacterium tumefaciens is the causative agent of crown gall, a plant tumour that can arise on most species of dicotyledonous plants. The tumour-inducing capacity of the bacterium requires the presence of a large plasmid, designated the Ti plasmid, which itself contains two regions essential for tumour formation-the T(umour)-region and the Vir(ulence)-region. The T-region is transferred to plant cells by an unknown mechanism, and becomes stably integrated into the plant genome. The Vir-region has been identified by transposon mutagenesis, but the DNA of this region has never been detected in tumour lines. However, trans-complementation of Vir mutants indicates that genes of the Vir-region are functional in the bacterium. Moreover, the Vir- and T-regions can be physically separated in A. tumefaciens without loss of tumour-inducing capacity. Seven loci, designated virA-F and virO, have been identified in the Vir-region of the octopine Ti plasmid, but their functions are unknown. As virC mutants in the octopine-type plasmid pTiB6 are invariably avirulent in tests on various plant species, this gene seems to be essential for virulence and we are studying it in detail. We report here that the promoter of virC shows no detectable activity in A. tumefaciens and Escherichia coli K-12 grown in standard medium, but that its activity is induced by a plant product.