353 research outputs found

    Using informative behavior to increase engagement while learning from human reward

    Get PDF
    In this work, we address a relatively unexplored aspect of designing agents that learn from human reward. We investigate how an agent’s non-task behavior can affect a human trainer’s training and agent learning. We use the TAMER framework, which facilitates the training of agents by human-generated reward signals, i.e., judgements of the quality of the agent’s actions, as the foundation for our investigation. Then, starting from the premise that the interaction between the agent and the trainer should be bi-directional, we propose two new training interfaces to increase a human trainer’s active involvement in the training process and thereby improve the agent’s task performance. One provides information on the agent’s uncertainty which is a metric calculated as data coverage, the other on its performance. Our results from a 51-subject user study show that these interfaces can induce the trainers to train longer and give more feedback. The agent’s performance, however, increases only in response to the addition of performance-oriented information, not by sharing uncertainty levels. These results suggest that the organizational maxim about human behavior, “you get what you measure”—i.e., sharing metrics with people causes them to focus on optimizing those metrics while de-emphasizing other objectives—also applies to the training of agents. Using principle component analysis, we show how trainers in the two conditions train agents differently. In addition, by simulating the influence of the agent’s uncertainty–informative behavior on a human’s training behavior, we show that trainers could be distracted by the agent sharing its uncertainty levels about its actions, giving poor feedback for the sake of reducing the agent’s uncertainty without improving the agent’s performance

    Temporal Stability and the Effects of Training on Saccade Latency in “Express Saccade Makers”

    Get PDF
    The temporal stability of saccade latency, and the effects of training, particularly in “express saccade makers” (ESMs), has received little attention. ESMs are healthy, naïve, adults, who persist in executing very many low latency “express saccades” (ES; saccades with latency of 80 ms to 130 ms), in conditions designed to suppress such responses. We investigated the stability of ES production (%ES) in 59 ESM and 54 non-ESM participants in overlap tasks. Within a single session, the intraclass correlation coefficient (ICC) for %ES in two runs of 200 trials was 0.97 (p30% of saccades over the two runs were ES, were classified as ESMs. For 60 participants tested over two sessions 12 weeks apart, and 30 participants tested in three sessions over approximately six months, the ICC for %ES was uniformly high (0.95, p<0.001 and 0.97, p<0.001 respectively) and participants behaved consistently with their initial classification. Fourteen participants (7 ESMs) were then exposed to training consisting of either gap or overlap tasks. Training increased %ES in both groups. However, when tested in overlap tasks, it was not sufficient to transform Normal participants into ESMs. We conclude that the pattern of saccade behaviour exhibited by ESMs constitutes a stable and distinct oculomotor phenotype

    Will all scientists working on snails and the diseases they transmit please stand up?

    Get PDF
    Copyright © 2012 Adema et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.No abstract available

    Genomic-Bioinformatic Analysis of Transcripts Enriched in the Third-Stage Larva of the Parasitic Nematode Ascaris suum

    Get PDF
    Differential transcription in Ascaris suum was investigated using a genomic-bioinformatic approach. A cDNA archive enriched for molecules in the infective third-stage larva (L3) of A. suum was constructed by suppressive-subtractive hybridization (SSH), and a subset of cDNAs from 3075 clones subjected to microarray analysis using cDNA probes derived from RNA from different developmental stages of A. suum. The cDNAs (n = 498) shown by microarray analysis to be enriched in the L3 were sequenced and subjected to bioinformatic analyses using a semi-automated pipeline (ESTExplorer). Using gene ontology (GO), 235 of these molecules were assigned to ‘biological process’ (n = 68), ‘cellular component’ (n = 50), or ‘molecular function’ (n = 117). Of the 91 clusters assembled, 56 molecules (61.5%) had homologues/orthologues in the free-living nematodes Caenorhabditis elegans and C. briggsae and/or other organisms, whereas 35 (38.5%) had no significant similarity to any sequences available in current gene databases. Transcripts encoding protein kinases, protein phosphatases (and their precursors), and enolases were abundantly represented in the L3 of A. suum, as were molecules involved in cellular processes, such as ubiquitination and proteasome function, gene transcription, protein–protein interactions, and function. In silico analyses inferred the C. elegans orthologues/homologues (n = 50) to be involved in apoptosis and insulin signaling (2%), ATP synthesis (2%), carbon metabolism (6%), fatty acid biosynthesis (2%), gap junction (2%), glucose metabolism (6%), or porphyrin metabolism (2%), although 34 (68%) of them could not be mapped to a specific metabolic pathway. Small numbers of these 50 molecules were predicted to be secreted (10%), anchored (2%), and/or transmembrane (12%) proteins. Functionally, 17 (34%) of them were predicted to be associated with (non-wild-type) RNAi phenotypes in C. elegans, the majority being embryonic lethality (Emb) (13 types; 58.8%), larval arrest (Lva) (23.5%) and larval lethality (Lvl) (47%). A genetic interaction network was predicted for these 17 C. elegans orthologues, revealing highly significant interactions for nine molecules associated with embryonic and larval development (66.9%), information storage and processing (5.1%), cellular processing and signaling (15.2%), metabolism (6.1%), and unknown function (6.7%). The potential roles of these molecules in development are discussed in relation to the known roles of their homologues/orthologues in C. elegans and some other nematodes. The results of the present study provide a basis for future functional genomic studies to elucidate molecular aspects governing larval developmental processes in A. suum and/or the transition to parasitism

    Annotation of two large contiguous regions from the Haemonchus contortus genome using RNA-seq and comparative analysis with Caenorhabditis elegans

    Get PDF
    The genomes of numerous parasitic nematodes are currently being sequenced, but their complexity and size, together with high levels of intra-specific sequence variation and a lack of reference genomes, makes their assembly and annotation a challenging task. Haemonchus contortus is an economically significant parasite of livestock that is widely used for basic research as well as for vaccine development and drug discovery. It is one of many medically and economically important parasites within the strongylid nematode group. This group of parasites has the closest phylogenetic relationship with the model organism Caenorhabditis elegans, making comparative analysis a potentially powerful tool for genome annotation and functional studies. To investigate this hypothesis, we sequenced two contiguous fragments from the H. contortus genome and undertook detailed annotation and comparative analysis with C. elegans. The adult H. contortus transcriptome was sequenced using an Illumina platform and RNA-seq was used to annotate a 409 kb overlapping BAC tiling path relating to the X chromosome and a 181 kb BAC insert relating to chromosome I. In total, 40 genes and 12 putative transposable elements were identified. 97.5% of the annotated genes had detectable homologues in C. elegans of which 60% had putative orthologues, significantly higher than previous analyses based on EST analysis. Gene density appears to be less in H. contortus than in C. elegans, with annotated H. contortus genes being an average of two-to-three times larger than their putative C. elegans orthologues due to a greater intron number and size. Synteny appears high but gene order is generally poorly conserved, although areas of conserved microsynteny are apparent. C. elegans operons appear to be partially conserved in H. contortus. Our findings suggest that a combination of RNA-seq and comparative analysis with C. elegans is a powerful approach for the annotation and analysis of strongylid nematode genomes

    Copper deficiency and effects of copper supplementation in a herd of red deer (Cervus elaphus)

    Get PDF
    Copper (Cu) deficiency was diagnosed in a Norwegian red deer (Cervus elaphus) herd subsequent to deaths due to emaciation in late autumn 1999. The animals had free access to salt licks containing 3000 mg Cu/kg. An evaluation of the herd revealed poor calf growth rate, low weights of adult hinds, dull and light-coloured hair coats and cases of diarrhoea. The herd was subsequently monitored throughout a three-year period of Cu-supplementation. The monitoring regimen included clinical observation, copper serum examination, weighing, faecal parasitological examination, and reproduction control by ultrasound. During the period January 2000 to May 2001, the animals were treated with Cu oxid capsules (1 g CuO/10 kg liveweight) at 2–4 months intervals, with the exception of March to September 2000. The animals were fed continuously with Cu-enriched concentrates containing 300 mg Cu/kg, at a rate of 1/2 kg per head and day, from May 2001 to January 2003. Following both copper supplementation regimens adequate serum Cu concentrations were measured, and markedly improved body weights, coat quality and reproductive results were observed, except for the period from March to September 2000 when no treatment was given. The results showed that in a deer herd, with a diet low in Cu, supplementation with CuO capsules had to be given at intervals of a few months to maintain adequate serum Cu levels. Free access to Cu-containing salt licks did not meet the animals' Cu demand. Good and stable results were achieved by the daily feeding of Cu-enriched concentrates

    Helminth secretions induce de novo T cell Foxp3 expression and regulatory function through the TGF-β pathway

    Get PDF
    Foxp3-expressing regulatory T (T reg) cells have been implicated in parasite-driven inhibition of host immunity during chronic infection. We addressed whether parasites can directly induce T reg cells. Foxp3 expression was stimulated in naive Foxp3⁻ T cells in mice infected with the intestinal helminth Heligmosomoides polygyrus. In vitro, parasite-secreted proteins (termed H. polygyrus excretory-secretory antigen [HES]) induced de novo Foxp3 expression in fluorescence-sorted Foxp3⁻ splenocytes from Foxp3-green fluorescent protein reporter mice. HES-induced T reg cells suppressed both in vitro effector cell proliferation and in vivo allergic airway inflammation. HES ligated the transforming growth factor (TGF) β receptor and promoted Smad2/3 phosphorylation. Foxp3 induction by HES was lost in dominant-negative TGF-βRII cells and was abolished by the TGF-β signaling inhibitor SB431542. This inhibitor also reduced worm burdens in H. polygyrus-infected mice. HES induced IL-17 in the presence of IL-6 but did not promote Th1 or Th2 development under any conditions. Importantly, antibody to mammalian TGF-β did not recognize HES, whereas antisera that inhibited HES did not affect TGF-β. Foxp3 was also induced by secreted products of Teladorsagia circumcincta, a related nematode which is widespread in ruminant animals. We have therefore identified a novel pathway through which helminth parasites may stimulate T reg cells, which is likely to be a key part of the parasite's immunological relationship with the host.J.R. Grainger thanks the Wellcome Trust for studentship support through the 4-year PhD Program, H.J. McSorley, K.J. Filbey, and C.A.M. Finney thank the Medical Research Council for studentship support, E.J.D. Greenwood thanks the Wellcome Trust for an undergraduate summer studentship, and K.A. Smith, J.P. Hewitson, Y. Harcus, and R.M. Maizels thank the Wellcome Trust for Programme Grant support. A.Y. Rudensky is a Howard Hughes Medical Institute Investigator and is supported by a National Institutes of Health grant
    corecore