401 research outputs found
Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
Reinforcement learning (RL) has shown great promise with algorithms learning
in environments with large state and action spaces purely from scalar reward
signals. A crucial challenge for current deep RL algorithms is that they
require a tremendous amount of environment interactions for learning. This can
be infeasible in situations where such interactions are expensive; such as in
robotics. Offline RL algorithms try to address this issue by bootstrapping the
learning process from existing logged data without needing to interact with the
environment from the very beginning. While online RL algorithms are typically
evaluated as a function of the number of environment interactions, there exists
no single established protocol for evaluating offline RL methods.In this paper,
we propose a sequential approach to evaluate offline RL algorithms as a
function of the training set size and thus by their data efficiency. Sequential
evaluation provides valuable insights into the data efficiency of the learning
process and the robustness of algorithms to distribution changes in the dataset
while also harmonizing the visualization of the offline and online learning
phases. Our approach is generally applicable and easy to implement. We compare
several existing offline RL algorithms using this approach and present insights
from a variety of tasks and offline datasets.Comment: TMLR 202
Versatile Confocal Raman Imaging Microscope Built from Off-the-Shelf Opto-Mechanical Components
Confocal Raman microscopic (CRM) imaging has evolved to become a key tool for spatially resolved, compositional analysis and imaging, down to the ÎŒm-scale, and nowadays one may choose between numerous commercial instruments. That notwithstanding, situations may arise which exclude the use of a commercial instrument, e.g., if the analysis involves toxic or radioactive samples/environments; one may not wish to render an expensive instrument unusable for other uses, due to contamination. Therefore, custom-designed CRM instrumentationâbeing adaptable to hazardous conditions and providing operational flexibilityâmay be beneficial. Here, we describe a CRM setup, which is constructed nearly in its entirety from off-the-shelf optomechanical and optical components. The original aim was to develop a CRM suitable for the investigation of samples exposed to tritium. For increased flexibility, the CRM system incorporates optical fiber coupling to both the Raman excitation laser and the spectrometer. Lateral raster scans and axial profiling of samples are facilitated by the use of a motorized xyz-translation assembly. Besides the description of the construction and alignment of the CRM system, we also provide (i) the experimental evaluation of system performance (such as, e.g., spatial resolution) and (ii) examples of Raman raster maps and axial profiles of selected thin-film samples (such as, e.g., graphene sheets)
Accurate reference gas mixtures containing tritiated molecules: Their production and ramanâbased analysis
Highly accurate, quantitative analyses of mixtures of hydrogen isotopologuesâboth the stable species, H, D2, and HD, and the radioactive species, T, HT, and DTâare of great importance in fields as diverse as deuteriumâtritium fusion, neutrino mass measurements using tritium ÎČ-decay, or for photonuclear experiments in which hydrogenâdeuterium targets are used. In this publication we describe a production, handling, and analysis facility capable of fabricating well-defined gas samples, which may contain any of the stable and radioactive hydrogen isotopologues, with sub-percent accuracy for the relative species concentrations. The production is based on precise manometric gas mixing of H, D, and T. The heteronuclear isotopologues HD, HT, and DT are generated via controlled, in-line catalytic reaction or by ÎČ-induced self-equilibration, respectively. The analysis was carried out using an in-line intensity- and wavelength-calibrated Raman spectroscopy system. This allows for continuous monitoring of the composition of the circulating gas during the self-equilibration or catalytic evolution phases. During all procedures, effects, such as exchange reactions with wall materials, were considered with care. Together with measurement statistics, these and other systematic effects were included in the determination of composition uncertainties of the generated reference gas samples. Measurement and calibration accuracy at the level of 1% was achieved
The Generation and Analysis of Tritium-substituted Methane
An unavoidable category of molecular species in large-scale tritium
applications, such as nuclear fusion, are tritium-substituted hydrocarbons;
these form by radiochemical reactions in the presence of (circulating) tritium
and carbon (mainly from the steel of vessels and tubing). Tritiumsubstituted
methane species, CQ (with Q = H , D , T), are often the precursor for
higher-order reaction chains, and thus are of particular interest. Here we
describe the controlled production of CQ carried out in the CAPER facility
of the Tritium Laboratory Karlsruhe (TLK), exploiting catalytic reactions and
species-enrichment via the CAPER-integral permeator. CQ4 was generated in
substantial quantity (>1000 cm at ~850 mbar, with CQ - content of up to
~20 %). These samples were analyzed using laser Raman and mass spectrometry, to
determine the relative isotopologue composition and to trace the generation of
tritiated chain-hydrocarbons.Comment: 10 pages, 4 figures. This article has been accepted for publication
in Fusion Science and Technology, published by Taylor & Franci
A combined analysis of short-baseline neutrino experiments in the (3+1) and (3+2) sterile neutrino oscillation hypotheses
We investigate adding two sterile neutrinos to resolve the apparent tension
existing between short-baseline neutrino oscillation results and
CPT-conserving, four-neutrino oscillation models. For both (3+1) and (3+2)
models, the level of statistical compatibility between the combined dataset
from the null short-baseline experiments Bugey, CHOOZ, CCFR84, CDHS, KARMEN,
and NOMAD, on the one hand; and the LSND dataset, on the other, is computed. A
combined analysis of all seven short-baseline experiments, including LSND, is
also performed, to obtain the favored regions in neutrino mass and mixing
parameter space for both models. Finally, four statistical tests to compare the
(3+1) and the (3+2) hypotheses are discussed. All tests show that (3+2) models
fit the existing short-baseline data significantly better than (3+1) models.Comment: 16 pages, 15 figures. Added NOMAD data to the analysis, one
statistical test, and two figures. References and text added. Version
submitted to PR
The KATRIN Pre-Spectrometer at reduced Filter Energy
The KArlsruhe TRItium Neutrino experiment, KATRIN, will determine the mass of
the electron neutrino with a sensitivity of 0.2 eV (90% C.L.) via a measurement
of the beta-spectrum of gaseous tritium near its endpoint of E_0 =18.57 keV. An
ultra-low background of about b = 10 mHz is among the requirements to reach
this sensitivity. In the KATRIN main beam-line two spectrometers of MAC-E
filter type are used in a tandem configuration. This setup, however, produces a
Penning trap which could lead to increased background. We have performed test
measurements showing that the filter energy of the pre-spectrometer can be
reduced by several keV in order to diminish this trap. These measurements were
analyzed with the help of a complex computer simulation, modeling multiple
electron reflections both from the detector and the photoelectric electron
source used in our test setup.Comment: 22 pages, 12 figure
Mapping the EQ-5D index by UPDRS and PDQ-8 in patients with Parkinsonâs disease
Background: Clinical studies employ the Unified Parkinsonâs Disease Rating Scale (UPDRS) to measure the severity of Parkinsonâs disease. Evaluations often fail to consider the health-related quality of life (HrQoL) or apply disease-specific instruments. Health-economic studies normally use estimates of utilities to calculate quality-adjusted life years. We aimed to develop an estimation algorithm for EuroQol- 5 dimensions (EQ-5D)-based utilities from the clinical UPDRS or disease-specific HrQoL data in the absence of original utilities estimates. Methods: Linear and fractional polynomial regression analyses were performed with data from a study of Parkinsonâs disease patients (n=138) to predict the EQ-5D index values from UPDRS and Parkinsonâs disease questionnaire eight dimensions (PDQ-8) data. German and European weights were used to calculate the EQ-5D index. The models were compared by R2, the root mean square error (RMS), the Bayesian information criterion, and Pregibonâs link test. Three independent data sets validated the models. Results: The regression analyses resulted in a single best prediction model (R2: 0.713 and 0.684, RMS: 0.139 and 13.78 for indices with German and European weights, respectively) consisting of UPDRS subscores II, III, IVa-c as predictors. When the PDQ-8 items were utilised as independent variables, the model resulted in an R2 of 0.60 and 0.67. The independent data confirmed the prediction models. Conclusion: The best results were obtained from a model consisting of UPDRS subscores II, III, IVa-c. Although a good model fit was observed, primary EQ-5D data are always preferable. Further validation of the prediction algorithm within large, independent studies is necessary prior to its generalised use
First observation of tritium adsorption on graphene
In this work, we report on the first-ever studies of graphene exposed to
tritium gas in a controlled environment. The single layer graphene on
/Si substrate was exposed to 400 mbar of for a
total time of ~55 h. The resistivity of the graphene sample was measured
in-situ during tritium exposure using the Van der Pauw method. After the
exposure, the samples were scanned with a confocal Raman microscope to study
the effect of tritium on the graphene structure as well as the homogeneity of
spectral modifications. We found that the sheet resistance increases by three
orders of magnitude during the exposure. By Raman microscopy, we demonstrate
that the graphene film can withstand the bombardment from the beta-decay of
tritium, and primary and secondary ions. Additionally, the Raman spectra after
tritium exposure are comparable with previously observed results in
hydrogen-loading experiments carried out by other groups. By thermal annealing
we could demonstrate, using Raman spectral analysis, that the structural
changes were partially reversible. Considering all observations, we conclude
that the graphene film was at least partially tritiated during the tritium
exposure.Comment: Submitted to Nanoscale Advances (RSC), 14 pages, 4 figure
Recommended from our members
Improved Upper Limit on the Neutrino Mass from a Direct Kinematic Method by KATRIN.
We report on the neutrino mass measurement result from the first four-week science run of the Karlsruhe Tritium Neutrino experiment KATRIN in spring 2019. Beta-decay electrons from a high-purity gaseous molecular tritium source are energy analyzed by a high-resolution MAC-E filter. A fit of the integrated electron spectrum over a narrow interval around the kinematic end point at 18.57 keV gives an effective neutrino mass square value of (-1.0_{-1.1}^{+0.9})ââeV^{2}. From this, we derive an upper limit of 1.1 eV (90% confidence level) on the absolute mass scale of neutrinos. This value coincides with the KATRIN sensitivity. It improves upon previous mass limits from kinematic measurements by almost a factor of 2 and provides model-independent input to cosmological studies of structure formation
- âŠ