How to Stay Curious while avoiding Noisy TVs using Aleatoric Uncertainty Estimation

When extrinsic rewards are sparse, artificial agents struggle to explore an environment. Curiosity, implemented as an intrinsic reward for prediction errors, can improve exploration but it is known to fail when faced with action-dependent noise sources (‘noisy TVs’). In an attempt to make exploring agents robust to noisy TVs, we present a simple solution: aleatoric mapping agents (AMAs). AMAs are a novel form of curiosity that explicitly ascertain which state transitions of the environment are unpredictable, even if those dynamics are induced by the actions of the agent. This is achieved by generating separate forward predictions for the mean and aleatoric uncertainty of future states, with the aim of reducing intrinsic rewards for those transitions that are unpredictable. We demonstrate that in a range of environments AMAs are able to circumvent actiondependent stochastic traps that immobilise conventional curiosity driven agents. Furthermore, we demonstrate empirically that other common exploration approaches—previously thought to be immune to agent-induced randomness—can be trapped by stochastic dynamics. Code to reproduce our experiments is provided

A Simple Approach for State-Action Abstraction using a Learned MDP Homomorphism

Animals are able to rapidly infer from limited experience when sets of state action pairs have equivalent reward and transition dynamics. On the other hand, modern reinforcement learning systems must painstakingly learn through trial and error that sets of state action pairs are value equivalent -- requiring an often prohibitively large amount of samples from their environment. MDP homomorphisms have been proposed that reduce the observed MDP of an environment to an abstract MDP, which can enable more sample efficient policy learning. Consequently, impressive improvements in sample efficiency have been achieved when a suitable MDP homomorphism can be constructed a priori -- usually by exploiting a practioner's knowledge of environment symmetries. We propose a novel approach to constructing a homomorphism in discrete action spaces, which uses a partial model of environment dynamics to infer which state action pairs lead to the same state -- reducing the size of the state-action space by a factor equal to the cardinality of the action space. We call this method equivalent effect abstraction. In a gridworld setting, we demonstrate empirically that equivalent effect abstraction can improve sample efficiency in a model-free setting and planning efficiency for modelbased approaches. Furthermore, we show on cartpole that our approach outperforms an existing method for learning homomorphisms, while using 33x less training data.Comment: Previously Presented at the Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM) 202

Genetic interaction mapping informs integrative structure determination of protein complexes

Determining structures of protein complexes is crucial for understanding cellular functions. Here, we describe an integrative structure determination approach that relies on in vivo measurements of genetic interactions. We construct phenotypic profiles for point mutations crossed against gene deletions or exposed to environmental perturbations, followed by converting similarities between two profiles into an upper bound on the distance between the mutated residues. We determine the structure of the yeast histone H3-H4 complex based on similar to 500,000 genetic interactions of 350 mutants. We then apply the method to subunits Rpb1-Rpb2 of yeast RNA polymerase II and subunits RpoB-RpoC of bacterial RNA polymerase. The accuracy is comparable to that based on chemical cross-links; using restraints from both genetic interactions and cross-links further improves model accuracy and precision. The approach provides an efficient means to augment integrative structure determination with in vivo observations

CandidaDB: a genome database for Candida albicans pathogenomics

CandidaDB is a database dedicated to the genome of the most prevalent systemic fungal pathogen of humans, Candida albicans. CandidaDB is based on an annotation of the Stanford Genome Technology Center C.albicans genome sequence data by the European Galar Fungail Consortium. CandidaDB Release 2.0 (June 2004) contains information pertaining to Assembly 19 of the genome of C.albicans strain SC5314. The current release contains 6244 annotated entries corresponding to 130 tRNA genes and 5917 protein-coding genes. For these, it provides tentative functional assignments along with numerous pre-run analyses that can assist the researcher in the evaluation of gene function for the purpose of specific or large-scale analysis. CandidaDB is based on GenoList, a generic relational data schema and a World Wide Web interface that has been adapted to the handling of eukaryotic genomes. The interface allows users to browse easily through genome data and retrieve information. CandidaDB also provides more elaborate tools, such as pattern searching, that are tightly connected to the overall browsing system. As the C.albicans genome is diploid and still incompletely assembled, CandidaDB provides tools to browse the genome by individual supercontigs and to examine information about allelic sequences obtained from complementary contigs. CandidaDB is accessible at http://genolist.pasteur.fr/CandidaDB

CandidaDB: A genome database for Candida albicans pathogenomics

CandidaDB is a database dedicated to the genome of the most prevalent systemic fungal pathogen of humans, Candida albicans. CandidaDB is based on an annotation of the Stanford Genome Technology Center C.albicans genome sequence data by the European Galar Fungail Consortium. CandidaDB Release 2.0 (June 2004) contains information pertaining to Assembly 19 of the genome of C.albicans strain SC5314. The current release contains 6244 annotated entries corresponding to 130 tRNA genes and 5917 protein-coding genes. For these, it provides tentative functional assignments along with numerous pre-run analyses that can assist the researcher in the evaluation of gene function for the purpose of specific or large-scale analysis. CandidaDB is based on GenoList, a generic relational data schema and a World Wide Web interface that has been adapted to the handling of eukaryotic genomes. The interface allows users to browse easily through genome data and retrieve information. CandidaDB also provides more elaborate tools, such as pattern searching, that are tightly connected to the overall browsing system. As the C.albicans genome is diploid and still incompletely assembled, CandidaDB provides tools to browse the genome by individual supercontigs and to examine information about allelic sequences obtained from complementary contigs. CandidaDB is accessible at http://genolist.pasteur.fr/CandidaDB.Sequence data from C.albicans were obtained from the Stanford Genome Technology Center (http://www.sequence. stanford.edu/group/candida). Sequencing of C.albicans was accomplished with the support of the NIDR and the Burroughs Wellcome Fund. This work was supported by grants from the European Commission (QLK2-2000-00795; MCRTN-CT-2003-504148; ‘Galar Fungail Consortium’) to A.J.P.B., C.E., A.D., J.E., C.G., B.H., F.M.K., J.P.M. and R.S. and the Ministere de la Recherche et de la Technologie (PRFMMIP ‘Re´seau Infections Fongiques’) to C.E. and C.G. F.T. was supported by the Institut Pasteur Strategic Horizontal Program on Anopheles gambiae. N.M. was supported by a fellowship of the Junta de Castilla y Leon and by grants DGCYT (PM-98-0317 and BIO 2002-02124) to A.D. R.S. was supported in part by grants from the Spanish Ministerio de Ciencia y Tecnologia (BMC2003- 01023) and Agencia Valenciana de Ciencia i Tecnologia de la Generalitat Valenciana (Grupos 03/187)

Balancing end-to-end budgets of the Georges Bank ecosystem

Author Posting. © Elsevier, 2007. This is the author's version of the work. It is posted here by permission of Elsevier for personal use, not for redistribution. The definitive version was published in Progress In Oceanography 74 (2007): 423-448, doi:10.1016/j.pocean.2007.05.003.Oceanographic regimes on the continental shelf display a great range in the time scales of physical exchange, biochemical processes and trophic transfers. The close surface-to-seabed physical coupling at intermediate scales of weeks to months means that the open ocean simplification to a purely pelagic food web is inadequate. Top-down trophic depictions, starting from the fish populations, are insufficient to constrain a system involving extensive nutrient recycling at lower trophic levels and subject to physical forcing as well as fishing. These pelagic-benthic interactions are found on all continental shelves but are particularly important on the relatively shallow Georges Bank in the northwest Atlantic. We have generated budgets for the lower food web for three physical regimes (well mixed, transitional and stratified) and for three seasons (spring, summer and fall/winter). The calculations show that vertical mixing and lateral exchange between the three regimes are important for zooplankton production as well as for nutrient input. Benthic suspension feeders are an additional critical pathway for transfers to higher trophic levels. Estimates of production by mesozooplankton, benthic suspension feeders and deposit feeders, derived primarily from data collected during the GLOBEC years of 1995-1999, provide input to an upper food web. Diets of commercial fish populations are used to calculate food requirements in three fish categories, planktivores, benthivores and piscivores, for four decades, 1963-2002, between which there were major changes in the fish communities. Comparisons of inputs from the lower web with fish energetic requirements for plankton and benthos indicate that we obtained reasonable agreement for the last three decades, 1973 to 2002. However, for the first decade, the fish food requirements were significantly less than the inputs. This decade, 1963-1972, corresponds to a period characterized by a strong Labrador Current and lower nitrate levels at the shelf edge, demonstrating how strong bottom-up physical forcing may determine overall fish yields.The research was done under the aegis of the U.S.-GLOBEC Northwest Atlantic Georges Bank Study, a program sponsored jointly by the U.S. National Science Foundation and the U.S. National Oceanic and Atmospheric Administration. We acknowledge NOAA-CICOR award NA17RJ1233 (J.H. Steele), NSF awards OCE0217399 (D.J. Gifford), OCE0217122 (J.J. Bisagni) and OCE0217257 (M.E. Sieracki). W.T. Stockhausen was supported by the NOAA Sponsored Coastal Ocean Research Program

Candida albicans Scavenges Host Zinc via Pra1 during Endothelial Invasion

The ability of pathogenic microorganisms to assimilate essential nutrients from their hosts is critical for pathogenesis. Here we report endothelial zinc sequestration by the major human fungal pathogen, Candida albicans. We hypothesised that, analogous to siderophore-mediated iron acquisition, C. albicans utilises an extracellular zinc scavenger for acquiring this essential metal. We postulated that such a “zincophore” system would consist of a secreted factor with zinc-binding properties, which can specifically reassociate with the fungal cell surface. In silico analysis of the C. albicans secretome for proteins with zinc binding motifs identified the pH-regulated antigen 1 (Pra1). Three-dimensional modelling of Pra1 indicated the presence of at least two zinc coordination sites. Indeed, recombinantly expressed Pra1 exhibited zinc binding properties in vitro. Deletion of PRA1 in C. albicans prevented fungal sequestration and utilisation of host zinc, and specifically blocked host cell damage in the absence of exogenous zinc. Phylogenetic analysis revealed that PRA1 arose in an ancient fungal lineage and developed synteny with ZRT1 (encoding a zinc transporter) before divergence of the Ascomycota and Basidiomycota. Structural modelling indicated physical interaction between Pra1 and Zrt1 and we confirmed this experimentally by demonstrating that Zrt1 was essential for binding of soluble Pra1 to the cell surface of C. albicans. Therefore, we have identified a novel metal acquisition system consisting of a secreted zinc scavenger (“zincophore”), which reassociates with the fungal cell. Furthermore, functional similarities with phylogenetically unrelated prokaryotic systems indicate that syntenic zinc acquisition loci have been independently selected during evolution

Cryptococcal Cell Morphology Affects Host Cell Interactions and Pathogenicity

Cryptococcus neoformans is a common life-threatening human fungal pathogen. The size of cryptococcal cells is typically 5 to 10 µm. Cell enlargement was observed in vivo, producing cells up to 100 µm. These morphological changes in cell size affected pathogenicity via reducing phagocytosis by host mononuclear cells, increasing resistance to oxidative and nitrosative stress, and correlated with reduced penetration of the central nervous system. Cell enlargement was stimulated by coinfection with strains of opposite mating type, and ste3aΔ pheromone receptor mutant strains had reduced cell enlargement. Finally, analysis of DNA content in this novel cell type revealed that these enlarged cells were polyploid, uninucleate, and produced daughter cells in vivo. These results describe a novel mechanism by which C. neoformans evades host phagocytosis to allow survival of a subset of the population at early stages of infection. Thus, morphological changes play unique and specialized roles during infection