15,153 research outputs found
The Dreaming Variational Autoencoder for Reinforcement Learning Environments
Reinforcement learning has shown great potential in generalizing over raw
sensory data using only a single neural network for value optimization. There
are several challenges in the current state-of-the-art reinforcement learning
algorithms that prevent them from converging towards the global optima. It is
likely that the solution to these problems lies in short- and long-term
planning, exploration and memory management for reinforcement learning
algorithms. Games are often used to benchmark reinforcement learning algorithms
as they provide a flexible, reproducible, and easy to control environment.
Regardless, few games feature a state-space where results in exploration,
memory, and planning are easily perceived. This paper presents The Dreaming
Variational Autoencoder (DVAE), a neural network based generative modeling
architecture for exploration in environments with sparse feedback. We further
present Deep Maze, a novel and flexible maze engine that challenges DVAE in
partial and fully-observable state-spaces, long-horizon tasks, and
deterministic and stochastic problems. We show initial findings and encourage
further work in reinforcement learning driven by generative exploration.Comment: Best Student Paper Award, Proceedings of the 38th SGAI International
Conference on Artificial Intelligence, Cambridge, UK, 2018, Artificial
Intelligence XXXV, 201
Simple threshold rules solve explore/exploit tradeâoffs in a resource accumulation search task
How, and how well, do people switch between exploration and exploitation to search for and accumulate resources? We study the decision processes underlying such exploration/exploitation tradeâoffs using a novel card selection task that captures the common situation of searching among multiple resources (e.g., jobs) that can be exploited without depleting. With experience, participants learn to switch appropriately between exploration and exploitation and approach optimal performance. We model participants' behavior on this task with random, threshold, and sampling strategies, and find that a linear decreasing threshold rule best fits participants' results. Further evidence that participants use decreasing thresholdâbased strategies comes from reaction time differences between exploration and exploitation; however, participants themselves report nonâdecreasing thresholds. Decreasing threshold strategies that âfrontâloadâ exploration and switch quickly to exploitation are particularly effective in resource accumulation tasks, in contrast to optimal stopping problems like the Secretary Problem requiring longer exploration
Evolution of the Pairwise Peculiar Velocity Distribution Function in Lagrangian Perturbation Theory
The statistical distribution of the radial pairwise peculiar velocity of
galaxies is known to have an exponential form as implied by observations and
explicitly shown in N-body simulations. Here we calculate its statistical
distribution function using the Zel'dovich approximation assuming that the
primordial density fluctuations are Gaussian distributed. We show that the
exponential distribution is realized as a transient phenomena on megaparsec
scales in the standard cold-dark-matter model.Comment: 19 pages, 8 Postscript figures, AAS LaTe
Temperature dependence of surface reconstructions of Au on Pd(110)
Surface reconstructions of Au film on Pd(110) substrate are studied using a
local Einstein approximation to quasiharmonic theory with the Sutton-Chen
interatomic potential. Temperature dependent surface free energies for
different coverages and surface structures are calculated. Experimentally
observed transformations from to and
structures can be explained in the framework of this model. Also conditions for
Stranski-Krastanov growth mode are found to comply with experiments. The domain
of validity of the model neglecting mixing entropy is analyzed.Comment: 7 pages, REVTeX two-column format, 3 postscript figures available on
request from [email protected] To appear in Phys. Rev. Letter
The Effects of Electrical Degradation on the Microstructure of Metal Oxide Varistor
In this paper, the findings from systematic experiments with the purpose to determine the degradation effect of single and multiple current pulses on the microstructure of the Metal Oxide (MO) varistor are described. Six distribution class varistors from one manufacturer were used in these experiments. The first part of the paper describes the electrical condition after application of single and multiple lightning current pulses. The results (before and after each elevated current impulse test) of 1mA AC reference voltage and residual voltage are presented in this section. We have also investigated a new technique called âReturn Voltage Measurementâ for monitoring of the degradation in MO varistors. This is described in detail. The second part deals with the microstructure observations of MO varistors. The results of microstructural examination of impulse current on MO varsitors are examined in detail. This section also explains the relationship between the microstructural changes and electrical degradation of MO varistors
Damping of Electron Density Structures and Implications for Interstellar Scintillation
The forms of electron density structures in kinetic Alfven wave turbulence
are studied in connection with scintillation. The focus is on small scales cm where the Kinetic Alfv\'en wave (KAW) regime is active in
the interstellar medium. MHD turbulence converts to a KAW cascade, starting at
10 times the ion gyroradius and continuing to smaller scales. These scales are
inferred to dominate scintillation in the theory of Boldyrev et al. From
numerical solutions of a decaying kinetic Alfv\'en wave turbulence model,
structure morphology reveals two types of localized structures, filaments and
sheets, and shows that they arise in different regimes of resistive and
diffusive damping. Minimal resistive damping yields localized current filaments
that form out of Gaussian-distributed initial conditions. When resistive
damping is large relative to diffusive damping, sheet-like structures form. In
the filamentary regime, each filament is associated with a non-localized
magnetic and density structure, circularly symmetric in cross section. Density
and magnetic fields have Gaussian statistics (as inferred from Gaussian-valued
kurtosis) while density gradients are strongly non-Gaussian, more so than
current. This enhancement of non-Gaussian statistics in a derivative field is
expected since gradient operations enhance small-scale fluctuations. The
enhancement of density gradient kurtosis over current kurtosis is not obvious,
yet it suggests that modest fluctuation levels in electron density may yield
large scintillation events during pulsar signal propagation in the interstellar
medium. In the sheet regime the same statistical observations hold, despite the
absence of localized filamentary structures. Probability density functions are
constructed from statistical ensembles in both regimes, showing clear formation
of long, highly non-Gaussian tails
Estimating the cost-effectiveness of detecting cases of chronic hepatitis C infection on reception into prison
Background
In England and Wales where less than 1% of the population are Injecting drug users (IDUs), 97% of HCV reports are attributed to injecting drug use. As over 60% of the IDU population will have been imprisoned by the age of 30 years, prison may provide a good location in which to offer HCV screening and treatment. The aim of this work is to examine the cost effectiveness of a number of alternative HCV case-finding strategies on prison reception
Methods
A decision analysis model embedded in a model of the flow of IDUs through prison was used to estimate the cost effectiveness of a number of alternative case-finding strategies. The model estimates the average cost of identifying a new case of HCV from the perspective of the health care provider and how these estimates may evolve over time.
Results
The results suggest that administering verbal screening for a past positive HCV test and for ever having engaged in illicit drug use prior to the administering of ELISA and PCR tests can have a significant impact on the cost effectiveness of HCV case-finding strategies on prison reception; the discounted cost in 2017 being ÂŁ2,102 per new HCV case detected compared to ÂŁ3,107 when no verbal screening is employed.
Conclusion
The work here demonstrates the importance of targeting those individuals that have ever engaged in illicit drug use for HCV testing in prisons, these individuals can then be targeted for future intervention measures such as treatment or monitored to prevent future transmission
Large-Scale Image Processing with the ROTSE Pipeline for Follow-Up of Gravitational Wave Events
Electromagnetic (EM) observations of gravitational-wave (GW) sources would
bring unique insights into a source which are not available from either channel
alone. However EM follow-up of GW events presents new challenges. GW events
will have large sky error regions, on the order of 10-100 square degrees, which
can be made up of many disjoint patches. When searching such large areas there
is potential contamination by EM transients unrelated to the GW event.
Furthermore, the characteristics of possible EM counterparts to GW events are
also uncertain. It is therefore desirable to be able to assess the statistical
significance of a candidate EM counterpart, which can only be done by
performing background studies of large data sets. Current image processing
pipelines such as that used by ROTSE are not usually optimised for large-scale
processing. We have automated the ROTSE image analysis, and supplemented it
with a post-processing unit for candidate validation and classification. We
also propose a simple ad hoc statistic for ranking candidates as more likely to
be associated with the GW trigger. We demonstrate the performance of the
automated pipeline and ranking statistic using archival ROTSE data. EM
candidates from a randomly selected set of images are compared to a background
estimated from the analysis of 102 additional sets of archival images. The
pipeline's detection efficiency is computed empirically by re-analysis of the
images after adding simulated optical transients that follow typical light
curves for gamma-ray burst afterglows and kilonovae. We show that the automated
pipeline rejects most background events and is sensitive to simulated
transients to limiting magnitudes consistent with the limiting magnitude of the
images
Investigation of Diagnostic Techniques for Metal Oxide Surge Arresters
Gapless metal oxide surge arresters(MOSA)have been available in the market for many years since they were first introduced in the 1970's. The aim of this study is to investigate some reliable diagnostic techniques to assess the condition of a metal oxide surge arrester when subjected to severe lightning strikes in the field. A number of non-destructive and destructive diagnostic techniques for Metal Oxide Surge Arrester (MOSA) are discussed in this paper. The non-destructive techniques include the standard 1 mA reference voltage, lightning impulse discharge residual voltage and a number of modern diagnostics based on polarization methods: Return voltage and polarization/depolarization current measurements. In order to observe, analyze and correctly explain the degradation phenomena, a number of destructive techniques based on microstructure observation are also conducted. The techniques include optical microscopy, scanning electron microscopy, X-Ray diffraction and energy dispersive spectrometry. The single and multi-pulse currents of 8r20 s wave shape were used to artificially degrade the MOSA. The before and after diagnostic results of the non-destructive and destructive techniques are presented and interpreted to understand the aging mechanism in MOSA. The importance of modern non-destructive electrical diagnostics based on polarization methods is validated by test results and is highlighted in detail in this paper. Finally the correlation of the results of different diagnostic techniques with each other and with the results of standard techniques is discussed
- âŠ