Stochastic Reinforcement Learning

In reinforcement learning episodes, the rewards and punishments are often non-deterministic, and there are invariably stochastic elements governing the underlying situation. Such stochastic elements are often numerous and cannot be known in advance, and they have a tendency to obscure the underlying rewards and punishments patterns. Indeed, if stochastic elements were absent, the same outcome would occur every time and the learning problems involved could be greatly simplified. In addition, in most practical situations, the cost of an observation to receive either a reward or punishment can be significant, and one would wish to arrive at the correct learning conclusion by incurring minimum cost. In this paper, we present a stochastic approach to reinforcement learning which explicitly models the variability present in the learning environment and the cost of observation. Criteria and rules for learning success are quantitatively analyzed, and probabilities of exceeding the observation cost bounds are also obtained.Comment: AIKE 201

Solving the Klein-Gordon equation using Fourier spectral methods: A benchmark test for computer performance

The cubic Klein-Gordon equation is a simple but non-trivial partial differential equation whose numerical solution has the main building blocks required for the solution of many other partial differential equations. In this study, the library 2DECOMP&FFT is used in a Fourier spectral scheme to solve the Klein-Gordon equation and strong scaling of the code is examined on thirteen different machines for a problem size of 512^3. The results are useful in assessing likely performance of other parallel fast Fourier transform based programs for solving partial differential equations. The problem is chosen to be large enough to solve on a workstation, yet also of interest to solve quickly on a supercomputer, in particular for parametric studies. Unlike other high performance computing benchmarks, for this problem size, the time to solution will not be improved by simply building a bigger supercomputer.Comment: 10 page

Ultra-large bandwidth hollow-core guiding in all-silica Bragg fibers with nano-supports

We demonstrate a new class of hollow-core Bragg fibers that are composed of concentric cylindrical silica rings separated by nanoscale support bridges. We theoretically predict and experimentally observe hollow-core confinement over an octave frequency range. The bandwidth of bandgap guiding in this new class of Bragg fibers exceeds that of other hollow-core fibers reported in the literature. With only three rings of silica cladding layers, these Bragg fibers achieve propagation loss of the order of 1 dB/m.Comment: 9 pages including 5 figure

The Caviar software package for the astrometric reduction of Cassini ISS images: description and examples

N.J.C. is grateful to the Paris Observatory for funding as an invited researcher at the IMCCE. We thank the FP7-ESPaCE European program for funding under the agreement No. 263466. N.J.C. and C.D.M. thank the Science and Technology Facilities Council (Grant No. ST/P000622/1) for financial support. This work was also supported by the International Space Science Institute (ISSI)

Prosociality and hoarding amid the COVID-19 pandemic : a tale of four countries

The COVID-19 pandemic is an unprecedented public health crisis that poses a challenge to humanity. Drawing on the stress and coping literature, we argue that people around the world alleviate their anxiety and stress induced by the pandemic through both prosocial and 'self-interested' hoarding behaviours. This cross-cultural survey study examined the pushing (threat perception) and pulling (moral identity) factors that predicted prosocial acts and hoarding, and subsequently psychological well-being. Data were collected from 9 April to 14 May 2020 from 251 participants in the United Kingdom (UK), 268 in the United States (US), 197 in Germany (DE), and 200 in Hong Kong (HK). Whereas threat perception was associated positively with both prosocial acts and hoarding, benevolent moral identity was associated positively with the former but not the latter behaviour. We also observed cross-cultural differences, such that both effects were stronger in more individualistic (UK, US) countries than less individualistic (HK, DE) ones. The findings shed light on the prosocial vs. self-interested behavioural responses of people in different cultures towards the same pandemic crisis

GABA Receptors and the Pharmacology of Sleep

Current GABAergic sleep-promoting medications were developed pragmatically, without making use of the immense diversity of GABAA receptors. Pharmacogenetic experiments are leading to an understanding of the circuit mechanisms in the hypothalamus by which zolpidem and similar compounds induce sleep at α2βγ2-type GABAA receptors. Drugs acting at more selective receptor types, for example, at receptors containing the α2 and/or α3 subunits expressed in hypothalamic and brain stem areas, could in principle be useful as hypnotics/anxiolytics. A highly promising sleep-promoting drug, gaboxadol, which activates αβδ-type receptors failed in clinical trials. Thus, for the time being, drugs such as zolpidem, which work as positive allosteric modulators at GABAA receptors, continue to be some of the most effective compounds to treat primary insomnia