Reinforcement learning or active inference?

This paper questions the need for reinforcement learning or control theory when optimising behaviour. We show that it is fairly simple to teach an agent complicated and adaptive behaviours using a free-energy formulation of perception. In this formulation, agents adjust their internal states and sampling of the environment to minimize their free-energy. Such agents learn causal structure in the environment and sample it in an adaptive and self-supervised fashion. This results in behavioural policies that reproduce those optimised by reinforcement learning and dynamic programming. Critically, we do not need to invoke the notion of reward, value or utility. We illustrate these points by solving a benchmark problem in dynamic programming; namely the mountain-car problem, using active perception or inference under the free-energy principle. The ensuing proof-of-concept may be important because the free-energy formulation furnishes a unified account of both action and perception and may speak to a reappraisal of the role of dopamine in the brain

Carboxy-Terminal Truncation Activates glp-1 Protein to Specify Vulval Fates in Caenorhabditis elegans

The glp-1 and lin-12 genes encode homologous transmembrane proteins that may act as receptors for cell interactions during development. The glp-1 product is required for induction of germ-line proliferation and for embryogenesis. By contrast, lin-12 mediates somatic cell interactions, including those between the precursor cells that form the vulval hypodermis (VPCs). Here we analyse an unusual allele of glp-1, glp-1(q35), which displays a semidominant multivulva phenotype (Muv), as well as the typical recessive, loss-of-function Glp phenotypes (sterility and embryonic lethality). We find that the effects of glp-1(q35) on VPC development mimic those of dominant lin-12 mutations, even in the absence of lin-12 activity. The glp-1(q35) gene bears a nonsense mutation predicted to eliminate the 122 C-terminal amino acids, including a ProGluSerThr (PEST) sequence thought to destabilize proteins. We suggest that the carboxy terminus bears a negative regulatory domain which normally inactivates glp-1 in the VPCs. We propose that inappropriate glp-1(q35) activity can substitute for lin-12 to determine vulval fate, perhaps by driving the VPCs to proliferate

Bilobalide modulates serotonin-controlled behaviors in the nematode Caenorhabditis elegans

<p>Abstract</p> <p>Background</p> <p>Dysfunctions in the serotonergic system have been implicated in several neurological disorders such as depression. Elderly individuals who have been diagnosed with clinical depression show elevated cases of neurodegenerative diseases. This has led to suggestions that modulating the serotonin (5-HT) system could provide an alternative method to current therapies for alleviating these pathologies. The neuroprotective effects of bilobalide <it>in vitro </it>have been documented. We aim to determine whether bilobalide affects the 5-HT system in the nematode <it>C. elegans</it>. The wild type worms, as well as well-characterized 5-HT mutants, were fed with bilobalide in a range of concentrations, and several 5-HT controlled behaviors were tested.</p> <p>Results</p> <p>We observed that bilobalide significantly inhibited 5-HT-controlled egg-laying behavior in a dose-dependent manner, which was blocked in the 5-HT receptor mutants (<it>ser-4, mod-1</it>), but not in the 5-HT transporter (<it>mod-5</it>) or synthesis (<it>tph-1</it>) mutants. Bilobalide also potentiated a 5-HT-controlled, experience-dependent locomotory behavior, termed the enhanced slowing response in the wild type animals. However, this effect was fully blocked in 5-HT receptor <it>mod-1 </it>and dopamine defective <it>cat-2 </it>mutants, but only partially blocked in <it>ser-4 </it>mutants. We also demonstrated that acetylcholine transmission was inhibited in a transgenic <it>C. elegans </it>strain that constitutively expresses Aβ, and bilobalide did not significantly affect this inhibition.</p> <p>Conclusion</p> <p>These results suggest that bilobalide may modulate specific 5-HT receptor subtypes, which involves interplay with dopamine transmission. Additional studies for the function of bilobalide in neurotransmitter systems could aid in our understanding of its neuroprotective properties.</p

Variability in gene expression underlies incomplete penetrance

The phenotypic differences between individual organisms can often be ascribed to underlying genetic and environmental variation. However, even genetically identical organisms in homogeneous environments vary, indicating that randomness in developmental processes such as gene expression may also generate diversity. To examine the consequences of gene expression variability in multicellular organisms, we studied intestinal specification in the nematode Caenorhabditis elegans in which wild-type cell fate is invariant and controlled by a small transcriptional network. Mutations in elements of this network can have indeterminate effects: some mutant embryos fail to develop intestinal cells, whereas others produce intestinal precursors. By counting transcripts of the genes in this network in individual embryos, we show that the expression of an otherwise redundant gene becomes highly variable in the mutants and that this variation is subjected to a threshold, producing an ON/OFF expression pattern of the master regulatory gene of intestinal differentiation. Our results demonstrate that mutations in developmental networks can expose otherwise buffered stochastic variability in gene expression, leading to pronounced phenotypic variation.National Institutes of Health (U.S.). Pioneer AwardMathematical Sciences Postdoctoral Research Fellowships (DMS-0603392)National Institutes of Health (U.S.). Ruth L. Kirschstein National Research Service Award (5F32GM080966

Visualization of C. elegans transgenic arrays by GFP

BACKGROUND: Targeting the green fluorescent protein (GFP) via the E. coli lac repressor (LacI) to a specific DNA sequence, the lac operator (lacO), allows visualization of chromosomes in yeast and mammalian cells. In principle this method of visualization could be used for genetic mosaic analysis, which requires cell-autonomous markers that can be scored easily and at single cell resolution. The C. elegans lin-3 gene encodes an epidermal growth factor family (EGF) growth factor. lin-3 is expressed in the gonadal anchor cell and acts through LET-23 (transmembrane protein tyrosine kinase and ortholog of EGF receptor) to signal the vulval precursor cells to generate vulval tissue. lin-3 is expressed in the vulval cells later, and recent evidence raises the possibility that lin-3 acts in the vulval cells as a relay signal during vulval induction. It is thus of interest to test the site of action of lin-3 by mosaic analysis. RESULTS: We visualized transgenes in living C. elegans by targeting the green fluorescent protein (GFP) via the E. coli lac repressor (LacI) to a specific 256 sequence repeat of the lac operator (lacO) incorporated into transgenes. We engineered animals to express a nuclear-localized GFP-LacI fusion protein. C. elegans cells having a lacO transgene result in nuclear-localized bright spots (i.e., GFP-LacI bound to lacO). Cells with diffuse nuclear fluorescence correspond to unbound nuclear localized GFP-LacI. We detected chromosomes in living animals by chromosomally integrating the array of the lacO repeat sequence and visualizing the integrated transgene with GFP-LacI. This detection system can be applied to determine polyploidy as well as investigating chromosome segregation. To assess the GFP-LacI•lacO system as a marker for mosaic analysis, we conducted genetic mosaic analysis of the epidermal growth factor lin-3, expressed in the anchor cell. We establish that lin-3 acts in the anchor cell to induce vulva development, demonstrating this method's utility in detecting the presence of a transgene. CONCLUSION: The GFP-LacI•lacO transgene detection system works in C. elegans for visualization of chromosomes and extrachromosomal transgenes. It can be used as a marker for genetic mosaic analysis. The lacO repeat sequence as an extrachromosomal array becomes a valuable technique allowing rapid, accurate determination of spontaneous loss of the array, thereby allowing high-resolution mosaic analysis. The lin-3 gene is required in the anchor cell to induce the epidermal vulval precursors cells to undergo vulval development

Transcriptome dynamics of a broad host-range cyanophage and its hosts

Cyanobacteria are highly abundant in the oceans and are constantly exposed to lytic viruses. The T4-like cyanomyoviruses are abundant in the marine environment and have broad host-ranges relative to other cyanophages. It is currently unknown whether broad host-range phages specifically tailor their infection program for each host, or employ the same program irrespective of the host infected. Also unknown is how different hosts respond to infection by the same phage. Here we used microarray and RNA-seq analyses to investigate the interaction between the Syn9 T4-like cyanophage and three phylogenetically, ecologically and genomically distinct marine Synechococcus strains: WH7803, WH8102 and WH8109. Strikingly, Syn9 led a nearly identical infection and transcriptional program in all three hosts. Different to previous assumptions for T4-like cyanophages, three temporally regulated gene expression classes were observed. Furthermore, a novel regulatory element controlled early-gene transcription, and host-like promoters drove middle gene transcription, different to the regulatory paradigm for T4. Similar results were found for the P-TIM40 phage during infection of Prochlorococcus NATL2A. Moreover, genomic and metagenomic analyses indicate that these regulatory elements are abundant and conserved among T4-like cyanophages. In contrast to the near-identical transcriptional program employed by Syn9, host responses to infection involved host-specific genes primarily located in hypervariable genomic islands, substantiating islands as a major axis of phage-cyanobacteria interactions. Our findings suggest that the ability of broad host-range phages to infect multiple hosts is more likely dependent on the effectiveness of host defense strategies than on differential tailoring of the infection process by the phage

E4 ligase–specific ubiquitination hubs coordinate DNA double-strand-break repair and apoptosis

Multiple protein ubiquitination events at DNA double-strand breaks (DSBs) regulate damage recognition, signaling and repair. It has remained poorly understood how the repair process of DSBs is coordinated with the apoptotic response. Here, we identified the E4 ubiquitin ligase UFD-2 as a mediator of DNA-damage-induced apoptosis in a genetic screen in Caenorhabditis elegans. We found that, after initiation of homologous recombination by RAD-51, UFD-2 forms foci that contain substrate-processivity factors including the ubiquitin-selective segregase CDC-48 (p97), the deubiquitination enzyme ATX-3 (Ataxin-3) and the proteasome. In the absence of UFD-2, RAD-51 foci persist, and DNA damage-induced apoptosis is prevented. In contrast, UFD-2 foci are retained until recombination intermediates are removed by the Holliday-junction-processing enzymes GEN-1, MUS-81 or XPF-1. Formation of UFD-2 foci also requires proapoptotic CEP-1 (p53) signaling. Our findings establish a central role of UFD-2 in the coordination between the DNA-repair process and the apoptotic response

Transcriptional control in the prereplicative phase of T4 development

Control of transcription is crucial for correct gene expression and orderly development. For many years, bacteriophage T4 has provided a simple model system to investigate mechanisms that regulate this process. Development of T4 requires the transcription of early, middle and late RNAs. Because T4 does not encode its own RNA polymerase, it must redirect the polymerase of its host, E. coli, to the correct class of genes at the correct time. T4 accomplishes this through the action of phage-encoded factors. Here I review recent studies investigating the transcription of T4 prereplicative genes, which are expressed as early and middle transcripts. Early RNAs are generated immediately after infection from T4 promoters that contain excellent recognition sequences for host polymerase. Consequently, the early promoters compete extremely well with host promoters for the available polymerase. T4 early promoter activity is further enhanced by the action of the T4 Alt protein, a component of the phage head that is injected into E. coli along with the phage DNA. Alt modifies Arg265 on one of the two α subunits of RNA polymerase. Although work with host promoters predicts that this modification should decrease promoter activity, transcription from some T4 early promoters increases when RNA polymerase is modified by Alt. Transcription of T4 middle genes begins about 1 minute after infection and proceeds by two pathways: 1) extension of early transcripts into downstream middle genes and 2) activation of T4 middle promoters through a process called sigma appropriation. In this activation, the T4 co-activator AsiA binds to Region 4 of σ70, the specificity subunit of RNA polymerase. This binding dramatically remodels this portion of σ70, which then allows the T4 activator MotA to also interact with σ70. In addition, AsiA restructuring of σ70 prevents Region 4 from forming its normal contacts with the -35 region of promoter DNA, which in turn allows MotA to interact with its DNA binding site, a MotA box, centered at the -30 region of middle promoter DNA. T4 sigma appropriation reveals how a specific domain within RNA polymerase can be remolded and then exploited to alter promoter specificity

A TRPV Channel Modulates C. elegans Neurosecretion, Larval Starvation Survival, and Adult Lifespan

For most organisms, food is only intermittently available; therefore, molecular mechanisms that couple sensation of nutrient availability to growth and development are critical for survival. These mechanisms, however, remain poorly defined. In the absence of nutrients, newly hatched first larval (L1) stage Caenorhabditis elegans halt development and survive in this state for several weeks. We isolated mutations in unc-31, encoding a calcium-activated regulator of neural dense-core vesicle release, which conferred enhanced starvation survival. This extended survival was reminiscent of that seen in daf-2 insulin-signaling deficient mutants and was ultimately dependent on daf-16, which encodes a FOXO transcription factor whose activity is inhibited by insulin signaling. While insulin signaling modulates metabolism, adult lifespan, and dauer formation, insulin-independent mechanisms that also regulate these processes did not promote starvation survival, indicating that regulation of starvation survival is a distinct program. Cell-specific rescue experiments identified a small subset of primary sensory neurons where unc-31 reconstitution modulated starvation survival, suggesting that these neurons mediate perception of food availability. We found that OCR-2, a transient receptor potential vanilloid (TRPV) channel that localizes to the cilia of this subset of neurons, regulates peptide-hormone secretion and L1 starvation survival. Moreover, inactivation of ocr-2 caused a significant extension in adult lifespan. These findings indicate that TRPV channels, which mediate sensation of diverse noxious, thermal, osmotic, and mechanical stimuli, couple nutrient availability to larval starvation survival and adult lifespan through modulation of neural dense-core vesicle secretion

Serotonergic chemosensory neurons modify the <i>C. elegans</i> immune response by regulating G-protein signaling in epithelial cells

The nervous and immune systems influence each other, allowing animals to rapidly protect themselves from changes in their internal and external environment. However, the complex nature of these systems in mammals makes it difficult to determine how neuronal signaling influences the immune response. Here we show that serotonin, synthesized in Caenorhabditis elegans chemosensory neurons, modulates the immune response. Serotonin released from these cells acts, directly or indirectly, to regulate G-protein signaling in epithelial cells. Signaling in these cells is required for the immune response to infection by the natural pathogen Microbacterium nematophilum. Here we show that serotonin signaling suppresses the innate immune response and limits the rate of pathogen clearance. We show that C. elegans uses classical neurotransmitters to alter the immune response. Serotonin released from sensory neurons may function to modify the immune system in response to changes in the animal's external environment such as the availability, or quality, of food