Using informative behavior to increase engagement while learning from human reward

In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance

Annotation of two large contiguous regions from the Haemonchus contortus genome using RNA-seq and comparative analysis with Caenorhabditis elegans

The genomes of numerous parasitic nematodes are currently being sequenced, but their complexity and size, together with high levels of intra-specific sequence variation and a lack of reference genomes, makes their assembly and annotation a challenging task. Haemonchus contortus is an economically significant parasite of livestock that is widely used for basic research as well as for vaccine development and drug discovery. It is one of many medically and economically important parasites within the strongylid nematode group. This group of parasites has the closest phylogenetic relationship with the model organism Caenorhabditis elegans, making comparative analysis a potentially powerful tool for genome annotation and functional studies. To investigate this hypothesis, we sequenced two contiguous fragments from the H. contortus genome and undertook detailed annotation and comparative analysis with C. elegans. The adult H. contortus transcriptome was sequenced using an Illumina platform and RNA-seq was used to annotate a 409 kb overlapping BAC tiling path relating to the X chromosome and a 181 kb BAC insert relating to chromosome I. In total, 40 genes and 12 putative transposable elements were identified. 97.5% of the annotated genes had detectable homologues in C. elegans of which 60% had putative orthologues, significantly higher than previous analyses based on EST analysis. Gene density appears to be less in H. contortus than in C. elegans, with annotated H. contortus genes being an average of two-to-three times larger than their putative C. elegans orthologues due to a greater intron number and size. Synteny appears high but gene order is generally poorly conserved, although areas of conserved microsynteny are apparent. C. elegans operons appear to be partially conserved in H. contortus. Our findings suggest that a combination of RNA-seq and comparative analysis with C. elegans is a powerful approach for the annotation and analysis of strongylid nematode genomes

Radon and childhood cancer

British Journal of Cancer (2002) 87, 1336–1337. doi:10.1038/sj.bjc.6600671 www.bjcancer.co

Genomic-Bioinformatic Analysis of Transcripts Enriched in the Third-Stage Larva of the Parasitic Nematode Ascaris suum

Differential transcription in Ascaris suum was investigated using a genomic-bioinformatic approach. A cDNA archive enriched for molecules in the infective third-stage larva (L3) of A. suum was constructed by suppressive-subtractive hybridization (SSH), and a subset of cDNAs from 3075 clones subjected to microarray analysis using cDNA probes derived from RNA from different developmental stages of A. suum. The cDNAs (n = 498) shown by microarray analysis to be enriched in the L3 were sequenced and subjected to bioinformatic analyses using a semi-automated pipeline (ESTExplorer). Using gene ontology (GO), 235 of these molecules were assigned to ‘biological process’ (n = 68), ‘cellular component’ (n = 50), or ‘molecular function’ (n = 117). Of the 91 clusters assembled, 56 molecules (61.5%) had homologues/orthologues in the free-living nematodes Caenorhabditis elegans and C. briggsae and/or other organisms, whereas 35 (38.5%) had no significant similarity to any sequences available in current gene databases. Transcripts encoding protein kinases, protein phosphatases (and their precursors), and enolases were abundantly represented in the L3 of A. suum, as were molecules involved in cellular processes, such as ubiquitination and proteasome function, gene transcription, protein–protein interactions, and function. In silico analyses inferred the C. elegans orthologues/homologues (n = 50) to be involved in apoptosis and insulin signaling (2%), ATP synthesis (2%), carbon metabolism (6%), fatty acid biosynthesis (2%), gap junction (2%), glucose metabolism (6%), or porphyrin metabolism (2%), although 34 (68%) of them could not be mapped to a specific metabolic pathway. Small numbers of these 50 molecules were predicted to be secreted (10%), anchored (2%), and/or transmembrane (12%) proteins. Functionally, 17 (34%) of them were predicted to be associated with (non-wild-type) RNAi phenotypes in C. elegans, the majority being embryonic lethality (Emb) (13 types; 58.8%), larval arrest (Lva) (23.5%) and larval lethality (Lvl) (47%). A genetic interaction network was predicted for these 17 C. elegans orthologues, revealing highly significant interactions for nine molecules associated with embryonic and larval development (66.9%), information storage and processing (5.1%), cellular processing and signaling (15.2%), metabolism (6.1%), and unknown function (6.7%). The potential roles of these molecules in development are discussed in relation to the known roles of their homologues/orthologues in C. elegans and some other nematodes. The results of the present study provide a basis for future functional genomic studies to elucidate molecular aspects governing larval developmental processes in A. suum and/or the transition to parasitism

Slepian functions and their use in signal estimation and spectral analysis

It is a well-known fact that mathematical functions that are timelimited (or spacelimited) cannot be simultaneously bandlimited (in frequency). Yet the finite precision of measurement and computation unavoidably bandlimits our observation and modeling scientific data, and we often only have access to, or are only interested in, a study area that is temporally or spatially bounded. In the geosciences we may be interested in spectrally modeling a time series defined only on a certain interval, or we may want to characterize a specific geographical area observed using an effectively bandlimited measurement device. It is clear that analyzing and representing scientific data of this kind will be facilitated if a basis of functions can be found that are "spatiospectrally" concentrated, i.e. "localized" in both domains at the same time. Here, we give a theoretical overview of one particular approach to this "concentration" problem, as originally proposed for time series by Slepian and coworkers, in the 1960s. We show how this framework leads to practical algorithms and statistically performant methods for the analysis of signals and their power spectra in one and two dimensions, and on the surface of a sphere.Comment: Submitted to the Handbook of Geomathematics, edited by Willi Freeden, Zuhair M. Nashed and Thomas Sonar, and to be published by Springer Verla

Deletion of parasite immune modulatory sequences combined with immune activating signals enhances vaccine mediated protection against filarial nematodes

<p>Background: Filarial nematodes are tissue-dwelling parasites that can be killed by Th2-driven immune effectors, but that have evolved to withstand immune attack and establish chronic infections by suppressing host immunity. As a consequence, the efficacy of a vaccine against filariasis may depend on its capacity to counter parasite-driven immunomodulation.</p> <p>Methodology and Principal Findings: We immunised mice with DNA plasmids expressing functionally-inactivated forms of two immunomodulatory molecules expressed by the filarial parasite Litomosoides sigmodontis: the abundant larval transcript-1 (LsALT) and cysteine protease inhibitor-2 (LsCPI). The mutant proteins enhanced antibody and cytokine responses to live parasite challenge, and led to more leukocyte recruitment to the site of infection than their native forms. The immune response was further enhanced when the antigens were targeted to dendritic cells using a single chain Fv-αDEC205 antibody and co-administered with plasmids that enhance T helper 2 immunity (IL-4) and antigen-presenting cell recruitment (Flt3L, MIP-1α). Mice immunised simultaneously against the mutated forms of LsALT and LsCPI eliminated adult parasites faster and consistently reduced peripheral microfilaraemia. A multifactorial analysis of the immune response revealed that protection was strongly correlated with the production of parasite-specific IgG1 and with the numbers of leukocytes present at the site of infection.</p> <p>Conclusions: We have developed a successful strategy for DNA vaccination against a nematode infection that specifically targets parasite-driven immunosuppression while simultaneously enhancing Th2 immune responses and parasite antigen presentation by dendritic cells.</p&gt

Scalar and vector Slepian functions, spherical signal estimation and spectral analysis

It is a well-known fact that mathematical functions that are timelimited (or spacelimited) cannot be simultaneously bandlimited (in frequency). Yet the finite precision of measurement and computation unavoidably bandlimits our observation and modeling scientific data, and we often only have access to, or are only interested in, a study area that is temporally or spatially bounded. In the geosciences we may be interested in spectrally modeling a time series defined only on a certain interval, or we may want to characterize a specific geographical area observed using an effectively bandlimited measurement device. It is clear that analyzing and representing scientific data of this kind will be facilitated if a basis of functions can be found that are "spatiospectrally" concentrated, i.e. "localized" in both domains at the same time. Here, we give a theoretical overview of one particular approach to this "concentration" problem, as originally proposed for time series by Slepian and coworkers, in the 1960s. We show how this framework leads to practical algorithms and statistically performant methods for the analysis of signals and their power spectra in one and two dimensions, and, particularly for applications in the geosciences, for scalar and vectorial signals defined on the surface of a unit sphere.Comment: Submitted to the 2nd Edition of the Handbook of Geomathematics, edited by Willi Freeden, Zuhair M. Nashed and Thomas Sonar, and to be published by Springer Verlag. This is a slightly modified but expanded version of the paper arxiv:0909.5368 that appeared in the 1st Edition of the Handbook, when it was called: Slepian functions and their use in signal estimation and spectral analysi

Planck intermediate results. XLI. A map of lensing-induced B-modes

The secondary cosmic microwave background (CMB) $B$-modes stem from the post-decoupling distortion of the polarization $E$-modes due to the gravitational lensing effect of large-scale structures. These lensing-induced $B$-modes constitute both a valuable probe of the dark matter distribution and an important contaminant for the extraction of the primary CMB $B$-modes from inflation. Planck provides accurate nearly all-sky measurements of both the polarization $E$-modes and the integrated mass distribution via the reconstruction of the CMB lensing potential. By combining these two data products, we have produced an all-sky template map of the lensing-induced $B$-modes using a real-space algorithm that minimizes the impact of sky masks. The cross-correlation of this template with an observed (primordial and secondary) $B$-mode map can be used to measure the lensing $B$-mode power spectrum at multipoles up to $2000$. In particular, when cross-correlating with the $B$-mode contribution directly derived from the Planck polarization maps, we obtain lensing-induced $B$-mode power spectrum measurement at a significance level of $12\,\sigma$, which agrees with the theoretical expectation derived from the Planck best-fit $\Lambda$CDM model. This unique nearly all-sky secondary $B$-mode template, which includes the lensing-induced information from intermediate to small ($10\lesssim \ell\lesssim 1000$) angular scales, is delivered as part of the Planck 2015 public data release. It will be particularly useful for experiments searching for primordial $B$-modes, such as BICEP2/Keck Array or LiteBIRD, since it will enable an estimate to be made of the lensing-induced contribution to the measured total CMB $B$-modes.Comment: 20 pages, 12 figures; Accepted for publication in A&A; The B-mode map is part of the PR2-2015 Cosmology Products; available as Lensing Products in the Planck Legacy Archive http://pla.esac.esa.int/pla/#cosmology; and described in the 'Explanatory Supplement' https://wiki.cosmos.esa.int/planckpla2015/index.php/Specially_processed_maps#2015_Lensing-induced_B-mode_ma

First Neutrino Observations from the Sudbury Neutrino Observatory

The first neutrino observations from the Sudbury Neutrino Observatory are presented from preliminary analyses. Based on energy, direction and location, the data in the region of interest appear to be dominated by 8B solar neutrinos, detected by the charged current reaction on deuterium and elastic scattering from electrons, with very little background. Measurements of radioactive backgrounds indicate that the measurement of all active neutrino types via the neutral current reaction on deuterium will be possible with small systematic uncertainties. Quantitative results for the fluxes observed with these reactions will be provided when further calibrations have been completed.Comment: Latex, 7 pages, 10 figures, Invited paper at Neutrino 2000 Conference, Sudbury, Canada, June 16-21, 2000 to be published in the Proceeding