Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia

We continuously face the dilemma of choosing between actions that gather new information or actions that exploit existing knowledge. This “exploration-exploitation” trade-off depends on the environment: stability favors exploiting knowledge to maximize gains; volatility favors exploring new options and discovering new outcomes. Here we set out to reconcile recent evidence for dopamine’s involvement in the exploration-exploitation trade-off with the existing evidence for basal ganglia control of action selection, by testing the hypothesis that tonic dopamine in the striatum, the basal ganglia’s input nucleus, sets the current exploration-exploitation trade-off. We first advance the idea of interpreting the basal ganglia output as a probability distribution function for action selection. Using computational models of the full basal ganglia circuit, we showed that, under this interpretation, the actions of dopamine within the striatum change the basal ganglia’s output to favor the level of exploration or exploitation encoded in the probability distribution. We also found that our models predict striatal dopamine controls the exploration-exploitation trade-off if we instead read-out the probability distribution from the target nuclei of the basal ganglia, where their inhibitory input shapes the cortical input to these nuclei. Finally, by integrating the basal ganglia within a reinforcement learning model, we showed how dopamine’s effect on the exploration-exploitation trade-off could be measurable in a forced two-choice task. These simulations also showed how tonic dopamine can appear to affect learning while only directly altering the trade-off. Thus, our models support the hypothesis that changes in tonic dopamine within the striatum can alter the exploration-exploitation trade-off by modulating the output of the basal ganglia

Structural basis for the design of selective phosphodiesterase 4B inhibitors

AbstractPhosphodiesterase-4B (PDE4B) regulates the pro-inflammatory Toll Receptor –Tumor Necrosis Factor α (TNFα) pathway in monocytes, macrophages and microglial cells. As such, it is an important, although under-exploited molecular target for anti-inflammatory drugs. This is due in part to the difficulty of developing selective PDE4B inhibitors as the amino acid sequence of the PDE4 active site is identical in all PDE4 subtypes (PDE4A-D). We show that highly selective PDE4B inhibitors can be designed by exploiting sequence differences outside the active site. Specifically, PDE4B selectivity can be achieved by capture of a C-terminal regulatory helix, now termed CR3 (Control Region 3), across the active site in a conformation that closes access by cAMP. PDE4B selectivity is driven by a single amino acid polymorphism in CR3 (Leu674 in PDE4B1 versus Gln594 in PDE4D). The reciprocal mutations in PDE4B and PDE4D cause a 70–80 fold shift in selectivity. Our structural studies show that CR3 is flexible and can adopt multiple orientations and multiple registries in the closed conformation. The new co-crystal structure with bound ligand provides a guide map for the design of PDE4B selective anti-inflammatory drugs

Genetic Association of Phosphodiesterases With Human Cognitive Performance

Recent, large-scale, genome-wide association studies (GWAS) provide a first view of the genetic fine structure of cognitive performance in healthy individuals. These studies have pooled data from up to 1.1 million subjects based on simple measures of cognitive performance including educational attainment, self-reported math ability, highest math class taken, and pooled, normalized scores from cognitive tests. These studies now allow the genome-wide interrogation of genes and pathways for their potential impact on human cognitive performance. The phosphodiesterase (PDE) enzymes regulate key cyclic nucleotide signaling pathways. Many are expressed in the brain and have been the targets of CNS drug discovery. Genetic variation in PDE1C, PDE4B and PDE4D associates with multiple measures of human cognitive function. The large size of the human PDE4B and PDE4D genes allows genetic fine structure mapping to transcripts encoding dimeric (long) forms of the enzymes. Upstream and downstream effectors of the cAMP pathway modulated by PDE4D [adenylate cyclase 1 (ADCY1), ADCY8, PRKAR1A, CREB1, or CREBBP] did not show genetic association with cognitive performance, however, genetic association was seen with brain derived neurotrophic factor (BDNF), a gene whose expression is modulated by cAMP. Notably absent was genetic association in healthy subjects to targets of CNS drug discovery designed to improve cognition in disease states by the modulation of cholinergic [acetylcholinesterase (ACHE), choline acetyltransferase (CHAT), nicotinic alpha 7 acetylcholine receptor (CHRNA7)], serotonergic (HTR6), histaminergic (HRH3), or glutamatergic (GRM5) pathways. These new data provide a rationale for exploring the therapeutic benefit of selective inhibitors of PDE1C, PDE4B and PDE4D in CNS disorders affecting cognition

Capturing Dopaminergic Modulation and Bimodal Membrane Behaviour of Striatal Medium Spiny Neurons in Accurate, Reduced Models

Loss of dopamine from the striatum can cause both profound motor deficits, as in Parkinson's disease, and disrupt learning. Yet the effect of dopamine on striatal neurons remains a complex and controversial topic, and is in need of a comprehensive framework. We extend a reduced model of the striatal medium spiny neuron (MSN) to account for dopaminergic modulation of its intrinsic ion channels and synaptic inputs. We tune our D1 and D2 receptor MSN models using data from a recent large-scale compartmental model. The new models capture the input–output relationships for both current injection and spiking input with remarkable accuracy, despite the order of magnitude decrease in system size. They also capture the paired pulse facilitation shown by MSNs. Our dopamine models predict that synaptic effects dominate intrinsic effects for all levels of D1 and D2 receptor activation. We analytically derive a full set of equilibrium points and their stability for the original and dopamine modulated forms of the MSN model. We find that the stability types are not changed by dopamine activation, and our models predict that the MSN is never bistable. Nonetheless, the MSN models can produce a spontaneously bimodal membrane potential similar to that recently observed in vitro following application of NMDA agonists. We demonstrate that this bimodality is created by modelling the agonist effects as slow, irregular and massive jumps in NMDA conductance and, rather than a form of bistability, is due to the voltage-dependent blockade of NMDA receptors. Our models also predict a more pronounced membrane potential bimodality following D1 receptor activation. This work thus establishes reduced yet accurate dopamine-modulated models of MSNs, suitable for use in large-scale models of the striatum. More importantly, these provide a tractable framework for further study of dopamine's effects on computation by individual neurons

BRAHMS: Novel middleware for integrated systems computation

Biological computational modellers are becoming increasingly interested in building large, eclectic models, including components on many different computational substrates, both biological and non-biological. At the same time, the rise of the philosophy of embodied modelling is generating a need to deploy biological models as controllers for robots in real-world environments. Finally, robotics engineers are beginning to find value in seconding biomimetic control strategies for use on practical robots. Together with the ubiquitous desire to make good on past software development effort, these trends are throwing up new challenges of intellectual and technological integration (for example across scales, across disciplines, and even across time) - challenges that are unmet by existing software frameworks. Here, we outline these challenges in detail, and go on to describe a newly developed software framework, BRAHMS. that meets them. BRAHMS is a tool for integrating computational process modules into a viable, computable system: its generality and flexibility facilitate integration across barriers, such as those described above, in a coherent and effective way. We go on to describe several cases where BRAHMS has been successfully deployed in practical situations. We also show excellent performance in comparison with a monolithic development approach. Additional benefits of developing in the framework include source code self-documentation, automatic coarse-grained parallelisation, cross-language integration, data logging, performance monitoring, and will include dynamic load-balancing and 'pause and continue' execution. BRAHMS is built on the nascent, and similarly general purpose, model markup language, SystemML. This will, in future, also facilitate repeatability and accountability (same answers ten years from now), transparent automatic software distribution, and interfacing with other SystemML tools. (C) 2009 Elsevier Ltd. All rights reserved

Dominant immunosuppression of dendritic cell function by prostate-cancer-derived exosomes

Exosomes are a distinct population of extracellular vesicles of endocytic origin with a protein repertoire similar to the parent cell. Although tumour-derived exosomes harbour immunosuppressive characteristics, they also carry tumour antigens and thus potentially contribute to immune activation. The aim of this study was to examine the impact of prostate cancer exosomes on tumour antigen cross-presentation. DU145 cells, transduced with shRNA to knockdown Rab27a (DU145KD) that inhibits exosome secretion, triggered significantly stronger tumour-antigen-specific T cell responses when loaded onto dendritic cells (DC) than control DU145 cells. Enhanced T cell response was prevented by adding purified exogenous DU145 exosomes to DU145KD cells, demonstrating that the dominant effect of tumour exosomes is immunosuppression and not antigen delivery. CD8+ T cell responses were impaired via exosomal regulation of DC function; exosomes triggered the expression of CD73, an ecto-5-nucleotidase responsible for AMP to adenosine hydrolysis, on DC. CD73 induction on DC that constitutively express CD39 resulted in an ATP-dependent inhibition of TNFα- and IL-12-production. We identified exosomal prostaglandin E2 (PGE2) as a potential driver of CD73 induction, as inhibition of PGE2 receptors significantly reduced exosome-dependent CD73 induction. The results reveal a hitherto unknown suppression of DC function via exosomal PGE2, adding a new element to tumour exosome–immune cell cross-talk

Is there an integrative center in the vertebrate brain-stem? A robotic evaluation of a model of the reticular formation viewed as an action selection device

Neurobehavioral data from intact, decerebrate, and neonatal rats, suggests that the reticular formation provides a brainstem substrate for action selection in the vertebrate central nervous system. In this article, Kilmer, McCulloch and Blum’s (1969, 1997) landmark reticular formation model is described and re-evaluated, both in simulation and, for the first time, as a mobile robot controller. Particular model configurations are found to provide effective action selection mechanisms in a robot survival task using either simulated or physical robots. The model’s competence is dependent on the organization of afferents from model sensory systems, and a genetic algorithm search identified a class of afferent configurations which have long survival times. The results support our proposal that the reticular formation evolved to provide effective arbitration between innate behaviors and, with the forebrain basal ganglia, may constitute the integrative, ’centrencephalic’ core of vertebrate brain architecture. Additionally, the results demonstrate that the Kilmer et al. model provides an alternative form of robot controller to those usually considered in the adaptive behavior literature

Functional Analysis of Novel Polymorphisms in the Human SLCO1A2 Gene that Encodes the Transporter OATP1A2

The solute carrier organic anion transporting polypeptide 1A2 (OATP1A2, SLCO1A2) is implicated in the cellular influx of a number of drugs. We identified five novel single nucleotide polymorphisms (SNPs) in coding exons of the SLCO1A2 gene in a cohort of subjects: G550A, G553A, G673A, A775C, and G862A, that encoded the OATP1A2 variants E184K, D185N, V255I, T259P, and D288N, respectively. The function and expression of these variant transporters were assessed in HEK-293 cells. We found that the novel variants, E184K, D185N, T259P, and D288N, were associated with impaired estrone-3-sulfate, imatinib, and methotrexate transport (∼20-50% of wild-type control); function was retained by OATP1A2-V255I. From biotinylation assays, the decreased function of these variants was due, at least in part, to impaired plasma membrane expression. The four loss-of-function variants were studied further using mutagenesis to produce variants that encode residues with different charges or steric properties. From immunoblotting, the replacement of negatively charged residues at amino acid positions 184 and 185 impaired membrane expression, while either a positive or negative charge at residue 288 supported the correct membrane targeting of OATP1A2. Replacement of T259 with bulky residues disrupted transporter stability. From molecular models, E184, D185, and D288 were located near several charged residues such that intramolecular ionic interactions may stabilize the transporter structure. Individuals who carry these novel SNPs in the SLCO1A2 gene may be at risk from impaired efficacy or enhanced toxicity during treatment with drugs that are substrates for OATP1A2.This study was supported by grants from Cancer Council NSW and the Australian National Health and Medical Research Council. The generous gifts of imatinib and 14C-imatinib from Novartis are gratefully acknowledged

Global Night-Time Lights for Observing Human Activity

We present a concept for a small satellite mission to make systematic, global observations of night-time lights with spatial resolution suitable for discerning the extent, type and density of human settlements. The observations will also allow better understanding of fine scale fossil fuel CO2 emission distribution. The NASA Earth Science Decadal Survey recommends more focus on direct observations of human influence on the Earth system. The most dramatic and compelling observations of human presence on the Earth are the night light observations taken by the Defence Meteorological System Program (DMSP) Operational Linescan System (OLS). Beyond delineating the footprint of human presence, night light data, when assembled and evaluated with complementary data sets, can determine the fine scale spatial distribution of global fossil fuel CO2 emissions. Understanding fossil fuel carbon emissions is critical to understanding the entire carbon cycle, and especially the carbon exchange between terrestrial and oceanic systems