DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explainability in the field of visual analytics

Reactions to Shocks and Monetary Policy Regimes: Inflation Targeting Versus Flexible Currency Board in Ghana, South Africa and the WAEMU

The aim of this paper is to examine the monetary policy actions through which the central banks in the Sub-Saharan African countries have searched to eliminate the negative impacts of the shocks facing their economies. We compare two types of monetary policy regimes: a currency board regime (in the CFA zone countries) and an inflation targeting policy regime (in Ghana and South Africa). We compare the properties of both policies when the central banks respond to three negative shocks hitting the economies: a recessionary demand shock, a supply shock increasing inflation and a negative fiscal shock. We propose an FPAS model (forecasting and monetary policy analysis system) that extends the usual FPAS models used in the literature to evaluate the impact of several policies in response to different types of exogenous shocks. We find that both policies are inappropriate to help the economies exiting from the effects of negative demand shocks (the adjustment relies mainly on fiscal policy), both are essential when negative shocks to primary balance occur (fiscal policy aggravates the negative effects of the shocks), while inflation targeting dominates the currency board regime as a strategy to cope with positive shocks to inflation.http://deepblue.lib.umich.edu/bitstream/2027.42/133082/1/wp1062.pd

How Transferable are Reasoning Patterns in VQA?

Since its inception, Visual Question Answering (VQA) is notoriously known as a task, where models are prone to exploit biases in datasets to find shortcuts instead of performing high-level reasoning. Classical methods address this by removing biases from training data, or adding branches to models to detect and remove biases. In this paper, we argue that uncertainty in vision is a dominating factor preventing the successful learning of reasoning in vision and language problems. We train a visual oracle and in a large scale study provide experimental evidence that it is much less prone to exploiting spurious dataset biases compared to standard models. We propose to study the attention mechanisms at work in the visual oracle and compare them with a SOTA Transformer-based model. We provide an in-depth analysis and visualizations of reasoning patterns obtained with an online visualization tool which we make publicly available (https://reasoningpatterns.github.io). We exploit these insights by transferring reasoning patterns from the oracle to a SOTA Transformer-based VQA model taking standard noisy visual inputs via fine-tuning. In experiments we report higher overall accuracy, as well as accuracy on infrequent answers for each question type, which provides evidence for improved generalization and a decrease of the dependency on dataset biases

Frequency tuning of the whispering gallery modes of silica microspheres for CQED and spectroscopy

We have tuned the whispering gallery modes of a fused silica microresonator over nearly 1 nm at 800 nm, i.e. over 0.5 FSR or 10^6 linewidths of the resonator. This has been achieved by a new method based on the stretching of a two-stem microsphere. The devices described below will permit new Cavity-QED experiments with this high-Q optical resonator when it is desirable to optimize its coupling to emitters with given transition frequencies. The tuning capability demonstrated here is compatible with both UHV and low temperature operation, which should be useful for future experiments with laser cooled atoms or single quantum dots.Comment: ReVTeX, 4 pages, 3 figure

In Subfertile Couple, Abdominal Fat Loss in Men Is Associated with Improvement of Sperm Quality and Pregnancy: A Case-Series

International audienceBackground: The impact of overweight among men of reproductive-age may affect fertility. Abdominal fat, more than body mass index, is an indicator of higher metabolic risk, which seems to be involved in decreasing sperm quality. This study aims to assess the relationship between abdominal fat and sperm DNA fragmentation and the effect of abdominal fat loss, among 6 men in subfertile couples. Methods: Sperm DNA fragmentation, abdominal fat and metabolic and hormonal profiles were measured in the 6 men before and after dietary advices. Seminal oxidative stress and antioxidant markers were determined. Results: After several months of a lifestyle program, all 6 men lost abdominal fat (patient 1: loss of 3 points of abdominal fat, patient 2: loss of 3 points, patient 3: loss of 2 points, patient 4: loss of 1 point, patient 5: loss of 4 points and patient 6: loss of 13 points). At the same time, their rate of sperm DNA fragmentation decreased: 9.5% vs 31%, 24% vs 43%, 18% vs 47%, 26.3% vs 66%, 25.4% vs 35% and 1.7% vs 25%. Also, an improvement in both metabolic (significant decrease in triglycerides and total cholesterol; p = 0.0139) and hormonal (significant increase in testosterone/oestradiol ratio; p = 0.0139) blood profiles was observed after following the lifestyle program. In seminal plasma, the amount of SOD2 has significantly increased (p = 0.0139) while in parallel carbonylated proteins have decreased. Furthermore, all spouses got pregnant. All pregnancies were brought to term. Conclusion: This study shows specifically that sperm DNA fragmentation among men in subfertile couples could be affected by abdominal fat, but improvement of lifestyle factor may correct this alteration. The effect of specific abdominal fat loss on sperm quality needs further investigation. The reduction of oxidative stress may be a contributing factor

DNA Methylation Supports Intrinsic Epigenetic Memory in Mammalian Cells

We have investigated the role of DNA methylation in the initiation and maintenance of silenced chromatin in somatic mammalian cells. We found that a mutated transgene, in which all the CpG dinucleotides have been eliminated, underwent transcriptional silencing to the same extent as the unmodified transgene. These observations demonstrate that DNA methylation is not required for silencing. The silenced CpG-free transgene exhibited all the features of heterochromatin, including silencing of transcriptional activity, delayed DNA replication, lack of histone H3 and H4 acetylation, lack of H3-K4 methylation, and enrichment in tri-methyl-H3-K9. In contrast, when we tested for transgene reactivation using a Cre recombinase-mediated inversion assay, we observed a marked difference between a CpG-free and an unmodified transgene: the CpG-free transgene resumed transcription and did not exhibit markers of heterochromatin whereas the unmodified transgene remained silenced. These data indicate that methylation of CpG residues conferred epigenetic memory in this system. These results also suggest that replication delay, lack of histone H3 and H4 acetylation, H3-K4 methylation, and enrichment in tri-methyl-H3-K9 are not sufficient to confer epigenetic memory. We propose that DNA methylation within transgenes serves as an intrinsic epigenetic memory to permanently silence transgenes and prevent their reactivation

Natural age dispersion arising from the analysis of broken crystals, part II. Practical application to apatite (U-Th)/He thermochronometry

We describe a new numerical inversion approach to deriving thermal history information from a range of naturally dispersed single grain apatite (U-Th)/He ages. The approach explicitly exploits the information about the shape of the 4He diffusion profile within individual grains that is inherent in the pattern of dispersion that arises from the common and routine practice of analysing broken crystals. Additional dispersion arising from differences in grain size and in U and Th concentration of grains, and the resultant changes to helium diffusivity caused by differential accumulation and annealing of radiation damage, is explicitly included. In this approach we calculate the ingrowth and loss, due to both thermal diffusion and the effects of α-ejection, of helium over time using a finite cylinder geometry. Broken grains are treated explicitly as fragments of an initially larger crystal. The initial grain lengths, L0, can be treated as unknown parameters to be estimated, although this is computationally demanding. A practical solution to the problem of solving for the unknown initial grain lengths is to simply apply a constant and sufficiently long L0 value to each fragment. We found that a good value for L0 was given by the maximum fragment length plus two times the maximum radius of a given set of fragments. Currently whole crystals and fragments with one termination are taken into account. A set of numerical experiments using synthetic fragment ages generated for increasingly complex thermal histories, and including realistic amounts of random noise (5-15%), are presented and show that useful thermal history information can be extracted from datasets showing very large dispersion. These include experiments where dispersion arises only from fragmentation of a single grain (length 400μm and radius 75μm, c. 6-50% dispersion), including the effects of grain size variation (for spherical equivalent grain radii between 74-122 μm, c. 10-70% dispersion) and the combined effects of fragmentation, grain size and radiation damage (for eU between 5-150 ppm, c.10-107% dispersion). Additionally we show that if the spherical equivalent radius of a broken grain is used as a measure of the effective diffusion domain for thermal history inversions then this will likely lead to erroneous thermal histories being obtained in many cases. The viability of the new technique is demonstrated for a real data set of 25 single grain (U-Th)/He apatite ages obtained for a gabbro sample from the BK-1 (Bierkraal) borehole drilled through the Bushveld Complex in South Africa. The inversion produces a well constrained thermal history consistent with both the (U-Th)/He data and available fission track analysis data. The advantage of the new approach is that it can explicitly accommodate all the details of conventional schemes, such as the effects of temporally variable diffusivity, zonation of U and Th and arbitrary grain size variations, and it works equally effectively for whole or broken crystals, and for the most common situation where a mixture of both are analysed. For the routine application of the apatite (U-Th)/He thermochronometry technique with samples where whole apatite grains are rare our experiments indicate that 15-20 single grain analyses are typically required to characterise the age dispersion pattern of a sample. The experiments also suggest that picking very short crystal fragments as well as long fragments, or even deliberately breaking long crystals to maximise the age dispersion in some cases, would ensure the best constraints on the thermal history models. The inversion strategy described in this paper is likely also directly applicable to other thermochronometers, such as the apatite, rutile and titanite U-Pb systems, where the diffusion domain is approximated by the physical grain size

Deep generative modeling for single-cell transcriptomics.

Single-cell transcriptome measurements can reveal unexplored biological diversity, but they suffer from technical noise and bias that must be modeled to account for the resulting uncertainty in downstream analyses. Here we introduce single-cell variational inference (scVI), a ready-to-use scalable framework for the probabilistic representation and analysis of gene expression in single cells ( https://github.com/YosefLab/scVI ). scVI uses stochastic optimization and deep neural networks to aggregate information across similar cells and genes and to approximate the distributions that underlie observed expression values, while accounting for batch effects and limited sensitivity. We used scVI for a range of fundamental analysis tasks including batch correction, visualization, clustering, and differential expression, and achieved high accuracy for each task

Comparing Numerical Methods for Isothermal Magnetized Supersonic Turbulence

We employ simulations of supersonic super-Alfvenic turbulence decay as a benchmark test problem to assess and compare the performance of nine astrophysical MHD methods actively used to model star formation. The set of nine codes includes: ENZO, FLASH, KT-MHD, LL-MHD, PLUTO, PPML, RAMSES, STAGGER, and ZEUS. We present a comprehensive set of statistical measures designed to quantify the effects of numerical dissipation in these MHD solvers. We compare power spectra for basic fields to determine the effective spectral bandwidth of the methods and rank them based on their relative effective Reynolds numbers. We also compare numerical dissipation for solenoidal and dilatational velocity components to check for possible impacts of the numerics on small-scale density statistics. Finally, we discuss convergence of various characteristics for the turbulence decay test and impacts of various components of numerical schemes on the accuracy of solutions. We show that the best performing codes employ a consistently high order of accuracy for spatial reconstruction of the evolved fields, transverse gradient interpolation, conservation law update step, and Lorentz force computation. The best results are achieved with divergence-free evolution of the magnetic field using the constrained transport method, and using little to no explicit artificial viscosity. Codes which fall short in one or more of these areas are still useful, but they must compensate higher numerical dissipation with higher numerical resolution. This paper is the largest, most comprehensive MHD code comparison on an application-like test problem to date. We hope this work will help developers improve their numerical algorithms while helping users to make informed choices in picking optimal applications for their specific astrophysical problems.Comment: 17 pages, 5 color figures, revised version to appear in ApJ, 735, July 201