Explore to Generalize in Zero-Shot RL

We study zero-shot generalization in reinforcement learning-optimizing a policy on a set of training tasks to perform well on a similar but unseen test task. To mitigate overfitting, previous work explored different notions of invariance to the task. However, on problems such as the ProcGen Maze, an adequate solution that is invariant to the task visualization does not exist, and therefore invariance-based approaches fail. Our insight is that learning a policy that effectively $\textit{explores}$ the domain is harder to memorize than a policy that maximizes reward for a specific task, and therefore we expect such learned behavior to generalize well; we indeed demonstrate this empirically on several domains that are difficult for invariance-based approaches. Our $\textit{Explore to Generalize}$ algorithm (ExpGen) builds on this insight: we train an additional ensemble of agents that optimize reward. At test time, either the ensemble agrees on an action, and we generalize well, or we take exploratory actions, which generalize well and drive us to a novel part of the state space, where the ensemble may potentially agree again. We show that our approach is the state-of-the-art on tasks of the ProcGen challenge that have thus far eluded effective generalization, yielding a success rate of $83\%$ on the Maze task and $74\%$ on Heist with $200$ training levels. ExpGen can also be combined with an invariance based approach to gain the best of both worlds, setting new state-of-the-art results on ProcGen

Development of the Video Analysis Scale of Engagement (VASE) for people with advanced dementia [version 2; peer review: 2 approved with reservations]

Background: The current study sought to develop a valid, reliable and unobtrusive tablet computer-based observational measure to assess engagement of people with advanced dementia. The Video Analysis Scale of Engagement (VASE) was designed to enable the rating of moment-by-moment changes in engagement during an activity, which would be useful for both future research and current residential care. Methods: An initial version of the VASE was tested. Face validity and content validity were assessed to validate an operational definition of engagement and develop an acceptable protocol for the scale. Thirtyseven non-professional and professional volunteers were recruited to view and rate level of engagement in music activities using the VASE. Results: An inter-class coefficient (ICC) test gave a high level of rating agreement across professionals and non-professionals. However, the ICC results of within-professionals were mixed. Linear mixed modelling suggested that the types of interventions (active or passive music listening), the particular intervention session being rated, time period of video and the age of raters could affect the ratings. Conclusions: Results suggested that raters used the VASE in a dynamic fashion and that the measure was able to distinguish between interventions. Further investigation and adjustments are warranted for this to be considered a valid and reliable scale in the measurement of engagement of people with advanced dementia in a residential care setting

Development of the Video Analysis Scale of Engagement (VASE) for people with advanced dementia [version 3; peer review: 1 approved, 1 approved with reservations]

BACKGROUND: The current study sought to develop a valid, reliable and unobtrusive tablet computer-based observational measure to assess engagement of people with advanced dementia. The Video Analysis Scale of Engagement (VASE) was designed to enable the rating of moment-by-moment changes in engagement during an activity, which would be useful for both future research and current residential care. METHODS: An initial version of the VASE was tested. Face validity and content validity were assessed to validate an operational definition of engagement and develop an acceptable protocol for the scale. Thirty-seven non-professional and professional volunteers were recruited to view and rate level of engagement in music activities using the VASE. RESULTS: An inter-class coefficient (ICC) test gave a high level of rating agreement across professionals and non-professionals. However, the ICC results of within-professionals were mixed. Linear mixed modelling suggested that the types of interventions (active or passive music listening), the particular intervention session being rated, time period of video and the age of raters could affect the ratings. CONCLUSIONS: Results suggested that raters used the VASE in a dynamic fashion and that the measure was able to distinguish between interventions. Further investigation and adjustments are warranted for this to be considered a valid and reliable scale in the measurement of engagement of people with advanced dementia in a residential care setting

Dynamic screening of a localized hole during photoemission from a metal cluster

Recent advances in attosecond spectroscopy techniques have fueled the interest in the theoretical description of electronic processes taking place in the subfemtosecond time scale. Here we study the coupled dynamic screening of a localized hole and a photoelectron emitted from a metal cluster using a semi-classical model. Electron density dynamics in the cluster is calculated with Time-Dependent Density Functional Theory and the motion of the photoemitted electron is described classically. We show that the dynamic screening of the hole by the cluster electrons affects the motion of the photoemitted electron. At the very beginning of its trajectory, the photoemitted electron interacts with the cluster electrons that pile up to screen the hole. Within our model, this gives rise to a significant reduction of the energy lost by the photoelectron. Thus, this is a velocity dependent effect that should be accounted for when calculating the average losses suffered by photoemitted electrons in metals.Comment: 15 pages, 5 figure

ZPS: visualization of recent adaptive evolution of proteins

<p>Abstract</p> <p>Background</p> <p>Detection of adaptive amino acid changes in proteins under recent short-term selection is of great interest for researchers studying microevolutionary processes in microbial pathogens or any other biological species. However, independent occurrence of such point mutations within genetically diverse haplotypes makes it difficult to detect the selection footprint by using traditional molecular evolutionary analyses. The recently developed Zonal Phylogeny (ZP) has been shown to be a useful analytic tool for identifying the footprints of short-term positive selection. ZP separates protein-encoding genes into evolutionarily long-term (with silent diversity) and short-term (without silent diversity) categories, or zones, followed by statistical analysis to detect signs of positive selection in the short-term zone. However, successful broad application of ZP for analysis of large haplotype datasets requires automation of the relatively labor-intensive computational process.</p> <p>Results</p> <p>Here we present Zonal Phylogeny Software (ZPS), an application that describes the distribution of single nucleotide polymorphisms (SNPs) of synonymous (silent) and non-synonymous (replacement) nature along branches of the DNA tree for any given protein-coding gene locus. Based on this information, ZPS separates the protein variant haplotypes with silent variability (Primary zone) from those that have recently evolved from the Primary zone variants by amino acid changes (External zone). Further comparative analysis of mutational hot-spot frequencies and haplotype diversity between the two zones allows determination of whether the External zone haplotypes emerged under positive selection.</p> <p>Conclusions</p> <p>As a visualization tool, ZPS depicts the protein tree in a DNA tree, indicating the most parsimonious numbers of synonymous and non-synonymous changes along the branches of a maximum-likelihood based DNA tree, along with information on homoplasy, reversion and structural mutation hot-spots. Through zonal differentiation, ZPS allows detection of recent adaptive evolution via selection of advantageous structural mutations, even when the advantage conferred by such mutations is relatively short-term (as in the case of "source-sink" evolutionary dynamics, which may represent a major mode of virulence evolution in microbes).</p

Quarkonium dissociation by anisotropy

We compute the screening length for quarkonium mesons moving through an anisotropic, strongly coupled N=4 super Yang-Mills plasma by means of its gravity dual. We present the results for arbitrary velocities and orientations of the mesons, as well as for arbitrary values of the anisotropy. The anisotropic screening length can be larger or smaller than the isotropic one, and this depends on whether the comparison is made at equal temperatures or at equal entropy densities. For generic motion we find that: (i) mesons dissociate above a certain critical value of the anisotropy, even at zero temperature; (ii) there is a limiting velocity for mesons in the plasma, even at zero temperature; (iii) in the ultra-relativistic limit the screening length scales as $(1-v^2)^\epsilon$ with \epsilon =1/2, in contrast with the isotropic result \epsilon =1/4.Comment: 39 pages, 26 figures; v2: minor changes, added reference

Jet quenching in a strongly coupled anisotropic plasma

The jet quenching parameter of an anisotropic plasma depends on the relative orientation between the anisotropic direction, the direction of motion of the parton, and the direction along which the momentum broadening is measured. We calculate the jet quenching parameter of an anisotropic, strongly coupled N=4 plasma by means of its gravity dual. We present the results for arbitrary orientations and arbitrary values of the anisotropy. The anisotropic value can be larger or smaller than the isotropic one, and this depends on whether the comparison is made at equal temperatures or at equal entropy densities. We compare our results to analogous calculations for the real-world quark-gluon plasma and find agreement in some cases and disagreement in others.Comment: 22 pages, 10 figures; v2: minor changes, added reference. Extends arXiv:1202.369

<i>Plasmodium falciparum </i>var genes expressed in children with severe malaria encode CIDRα1 domains

Most severe Plasmodium falciparum infections are experienced by young children. Severe symptoms are precipitated by vascular sequestration of parasites expressing a particular subset of the polymorphic P. falciparum erythrocyte membrane protein 1 (PfEMP1) adhesion molecules. Parasites binding human endothelial protein C receptor (EPCR) through the CIDRα1 domain of certain PfEMP1 were recently associated with severe malaria in children. However, it has remained unclear to which extend the EPCR‐binding CIDRα1 domains epitomize PfEMP1 expressed in severe malaria. Here, we characterized the near full‐length transcripts dominating the var transcriptome in children with severe malaria and found that the only common feature of the encoded PfEMP1 was CIDRα1 domains. Such genes were highly and dominantly expressed in both children with severe malarial anaemia and cerebral malaria. These observations support the hypothesis that the CIDRα1‐EPCR interaction is key to the pathogenesis of severe malaria and strengthen the rationale for pursuing a vaccine or adjunctive treatment aiming at inhibiting or reducing the damaging effects of this interaction

Drag force in a strongly coupled anisotropic plasma

We calculate the drag force experienced by an infinitely massive quark propagating at constant velocity through an anisotropic, strongly coupled N=4 plasma by means of its gravity dual. We find that the gluon cloud trailing behind the quark is generally misaligned with the quark velocity, and that the latter is also misaligned with the force. The drag coefficient $\mu$ can be larger or smaller than the corresponding isotropic value depending on the velocity and the direction of motion. In the ultra-relativistic limit we find that generically $\mu \propto p$. We discuss the conditions under which this behaviour may extend to more general situations.Comment: 25 pages, 13 figures; v2: minor changes, added reference