Complex Processes from Dynamical Architectures with Time-Scale Hierarchy

The idea that complex motor, perceptual, and cognitive behaviors are composed of smaller units, which are somehow brought into a meaningful relation, permeates the biological and life sciences. However, no principled framework defining the constituent elementary processes has been developed to this date. Consequently, functional configurations (or architectures) relating elementary processes and external influences are mostly piecemeal formulations suitable to particular instances only. Here, we develop a general dynamical framework for distinct functional architectures characterized by the time-scale separation of their constituents and evaluate their efficiency. Thereto, we build on the (phase) flow of a system, which prescribes the temporal evolution of its state variables. The phase flow topology allows for the unambiguous classification of qualitatively distinct processes, which we consider to represent the functional units or modes within the dynamical architecture. Using the example of a composite movement we illustrate how different architectures can be characterized by their degree of time scale separation between the internal elements of the architecture (i.e. the functional modes) and external interventions. We reveal a tradeoff of the interactions between internal and external influences, which offers a theoretical justification for the efficient composition of complex processes out of non-trivial elementary processes or functional modes

Complex Evolutionary History of the Aeromonas veronii Group Revealed by Host Interaction and DNA Sequence Data

Aeromonas veronii biovar sobria, Aeromonas veronii biovar veronii, and Aeromonas allosaccharophila are a closely related group of organisms, the Aeromonas veronii Group, that inhabit a wide range of host animals as a symbiont or pathogen. In this study, the ability of various strains to colonize the medicinal leech as a model for beneficial symbiosis and to kill wax worm larvae as a model for virulence was determined. Isolates cultured from the leech out-competed other strains in the leech model, while most strains were virulent in the wax worms. Three housekeeping genes, recA, dnaJ and gyrB, the gene encoding chitinase, chiA, and four loci associated with the type three secretion system, ascV, ascFG, aexT, and aexU were sequenced. The phylogenetic reconstruction failed to produce one consensus tree that was compatible with most of the individual genes. The Approximately Unbiased test and the Genetic Algorithm for Recombination Detection both provided further support for differing evolutionary histories among this group of genes. Two contrasting tests detected recombination within aexU, ascFG, ascV, dnaJ, and gyrB but not in aexT or chiA. Quartet decomposition analysis indicated a complex recent evolutionary history for these strains with a high frequency of horizontal gene transfer between several but not among all strains. In this study we demonstrate that at least for some strains, horizontal gene transfer occurs at a sufficient frequency to blur the signal from vertically inherited genes, despite strains being adapted to distinct niches. Simply increasing the number of genes included in the analysis is unlikely to overcome this challenge in organisms that occupy multiple niches and can exchange DNA between strains specialized to different niches. Instead, the detection of genes critical in the adaptation to specific niches may help to reveal the physiological specialization of these strains

Time Scale Hierarchies in the Functional Organization of Complex Behaviors

Traditional approaches to cognitive modelling generally portray cognitive events in terms of ‘discrete’ states (point attractor dynamics) rather than in terms of processes, thereby neglecting the time structure of cognition. In contrast, more recent approaches explicitly address this temporal dimension, but typically provide no entry points into cognitive categorization of events and experiences. With the aim to incorporate both these aspects, we propose a framework for functional architectures. Our approach is grounded in the notion that arbitrary complex (human) behaviour is decomposable into functional modes (elementary units), which we conceptualize as low-dimensional dynamical objects (structured flows on manifolds). The ensemble of modes at an agent’s disposal constitutes his/her functional repertoire. The modes may be subjected to additional dynamics (termed operational signals), in particular, instantaneous inputs, and a mechanism that sequentially selects a mode so that it temporarily dominates the functional dynamics. The inputs and selection mechanisms act on faster and slower time scales then that inherent to the modes, respectively. The dynamics across the three time scales are coupled via feedback, rendering the entire architecture autonomous. We illustrate the functional architecture in the context of serial behaviour, namely cursive handwriting. Subsequently, we investigate the possibility of recovering the contributions of functional modes and operational signals from the output, which appears to be possible only when examining the output phase flow (i.e., not from trajectories in phase space or time)

A computational psychiatry approach identifies how alpha-2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque

[EN] Noradrenaline is believed to support cognitive flexibility through the alpha 2A noradrenergic receptor (a2A-NAR) acting in prefrontal cortex. Enhanced flexibility has been inferred from improved working memory with the a2A-NA agonist Guanfacine. But it has been unclear whether Guanfacine improves specific attention and learning mechanisms beyond working memory, and whether the drug effects can be formalized computationally to allow single subject predictions. We tested and confirmed these suggestions in a case study with a healthy nonhuman primate performing a feature-based reversal learning task evaluating performance using Bayesian and Reinforcement learning models. In an initial dose-testing phase we found a Guanfacine dose that increased performance accuracy, decreased distractibility and improved learning. In a second experimental phase using only that dose we examined the faster feature-based reversal learning with Guanfacine with single-subject computational modeling. Parameter estimation suggested that improved learning is not accounted for by varying a single reinforcement learning mechanism, but by changing the set of parameter values to higher learning rates and stronger suppression of non-chosen over chosen feature information. These findings provide an important starting point for developing nonhuman primate models to discern the synaptic mechanisms of attention and learning functions within the context of a computational neuropsychiatry framework.This research was supported by grants from the Canadian Institutes of Health Research (CIHR), the Natural Sciences and Engineering Research Council of Canada (NSERC) and the Ontario Ministry of Economic Development and Innovation (MEDI). We thank Dr. Hongying Wang for invaluable help with drug administration and animal careHassani, SA.; Oemisch, M.; Balcarras, M.; Westendorff, S.; Ardid-Ramírez, JS.; Van Der Meer, MA.; Tiesinga, P.... (2017). A computational psychiatry approach identifies how alpha-2A noradrenergic agonist Guanfacine affects feature-based reinforcement learning in the macaque. Scientific Reports. 7:1-19. https://doi.org/10.1038/srep40606S1197Arnsten, A. F., Wang, M. J. & Paspalas, C. D. Neuromodulation of thought: flexibilities and vulnerabilities in prefrontal cortical network synapses. Neuron 76, 223–239 (2012).Arnsten, A. F. & Dudley, A. G. Methylphenidate improves prefrontal cortical cognitive function through alpha2 adrenoceptor and dopamine D1 receptor actions: Relevance to therapeutic effects in Attention Deficit Hyperactivity Disorder. Behav Brain Funct 1, 2 (2005).Clark, K. L. & Noudoost, B. The role of prefrontal catecholamines in attention and working memory. Front Neural Circuits 8, 33 (2014).Wang, M. et al. Neuronal basis of age-related working memory decline. Nature 476, 210–213 (2011).Wang, M. et al. Alpha2A-adrenoceptors strengthen working memory networks by inhibiting cAMP-HCN channel signaling in prefrontal cortex. Cell 129, 397–410 (2007).Aston-Jones, G. & Cohen, J. D. An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. Annu Rev Neurosci 28, 403–450 (2005).Yu, A. J. & Dayan, P. Uncertainty, neuromodulation, and attention. Neuron 46, 681–692 (2005).Mather, M., Clewett, D., Sakaki, M. & Harley, C. W. Norepinephrine ignites local hot spots of neuronal excitation: How arousal amplifies selectivity in perception and memory. Behav Brain Sci, 1–100, doi: 10.1017/S0140525X15000667 (2015).Amemiya, S. & Redish, A. D. Manipulating Decisiveness in Decision Making: Effects of Clonidine on Hippocampal Search Strategies. J Neurosci 36, 814–827 (2016).Doya, K. Metalearning and neuromodulation. Neural Netw 15, 495–506 (2002).Uhlen, S., Muceniece, R., Rangel, N., Tiger, G. & Wikberg, J. E. Comparison of the binding activities of some drugs on alpha 2A, alpha 2B and alpha 2C-adrenoceptors and non-adrenergic imidazoline sites in the guinea pig. Pharmacology & toxicology 76, 353–364 (1995).Mao, Z. M., Arnsten, A. F. & Li, B. M. Local infusion of an alpha-1 adrenergic agonist into the prefrontal cortex impairs spatial working memory performance in monkeys. Biological psychiatry 46, 1259–1265 (1999).Arnsten, A. F. & Goldman-Rakic, P. S. Analysis of alpha-2 adrenergic agonist effects on the delayed nonmatch-to-sample performance of aged rhesus monkeys. Neurobiol Aging 11, 583–590 (1990).Franowicz, J. S. & Arnsten, A. F. The alpha-2a noradrenergic agonist, guanfacine, improves delayed response performance in young adult rhesus monkeys. Psychopharmacology 136, 8–14 (1998).Caetano, M. S. et al. Noradrenergic control of error perseveration in medial prefrontal cortex. Frontiers in Integrative Neuroscience 6, 125 (2012).Kim, S., Bobeica, I., Gamo, N. J., Arnsten, A. F. & Lee, D. Effects of alpha-2A adrenergic receptor agonist on time and risk preference in primates. Psychopharmacology 219, 363–375 (2012).Seu, E., Lang, A., Rivera, R. J. & Jentsch, J. D. Inhibition of the norepinephrine transporter improves behavioral flexibility in rats and monkeys. Psychopharmacology 202, 505–519 (2009).Kawaura, K., Karasawa, J., Chaki, S. & Hikichi, H. Stimulation of postsynapse adrenergic alpha2A receptor improves attention/cognition performance in an animal model of attention deficit hyperactivity disorder. Behav Brain Res 270, 349–356 (2014).Aoki, C., Go, C. G., Venkatesan, C. & Kurose, H. Perikaryal and synaptic localization of alpha 2A-adrenergic receptor-like immunoreactivity. Brain Res 650, 181–204 (1994).Barth, A. M., Vizi, E. S., Zelles, T. & Lendvai, B. Alpha2-adrenergic receptors modify dendritic spike generation via HCN channels in the prefrontal cortex. J Neurophysiol 99, 394–401 (2008).Ji, X. H., Ji, J. Z., Zhang, H. & Li, B. M. Stimulation of alpha2-adrenoceptors suppresses excitatory synaptic transmission in the medial prefrontal cortex of rat. Neuropsychopharmacology 33, 2263–2271 (2008).Yi, F., Liu, S. S., Luo, F., Zhang, X. H. & Li, B. M. Signaling mechanism underlying alpha2A -adrenergic suppression of excitatory synaptic transmission in the medial prefrontal cortex of rats. Eur J Neurosci 38, 2364–2373 (2013).Engberg, G. & Eriksson, E. Effects of alpha 2-adrenoceptor agonists on locus coeruleus firing rate and brain noradrenaline turnover in N-ethoxycarbonyl-2-ethoxy-1,2-dihydroquinoline (EEDQ)-treated rats. Naunyn-Schmiedeberg’s archives of pharmacology 343, 472–477 (1991).Jakala, P. et al. Guanfacine, but not clonidine, improves planning and working memory performance in humans. Neuropsychopharmacology 20, 460–470 (1999).Jakala, P. et al. Guanfacine and clonidine, alpha 2-agonists, improve paired associates learning, but not delayed matching to sample, in humans. Neuropsychopharmacology 20, 119–130 (1999).Muller, U. et al. Lack of effects of guanfacine on executive and memory functions in healthy male volunteers. Psychopharmacology 182, 205–213 (2005).Scahill, L. et al. A placebo-controlled study of guanfacine in the treatment of children with tic disorders and attention deficit hyperactivity disorder. The American journal of psychiatry 158, 1067–1074 (2001).Huys, Q. J. M., Maia, T. V. & Frank, M. J. Computational psychiatry as a bridge from neuroscience to clinical applications. Nat Neurosci 19, 404–413 (2016).Stephan, K. E. et al. Computational neuroimaging strategies for single patient predictions. NeuroImage in press (2015).Arnsten, A. F., Cai, J. X. & Goldman-Rakic, P. S. The alpha-2 adrenergic agonist guanfacine improves memory in aged monkeys without sedative or hypotensive side effects: evidence for alpha-2 receptor subtypes. J Neurosci 8, 4287–4298 (1988).Callado, L. F. & Stamford, J. A. Alpha2A- but not alpha2B/C-adrenoceptors modulate noradrenaline release in rat locus coeruleus: voltammetric data. Eur J Pharmacol 366, 35–39 (1999).Millan, M. J. et al. Cognitive dysfunction in psychiatric disorders: characteristics, causes and the quest for improved therapy. Nature reviews. Drug discovery 11, 141–168 (2012).Niv, Y. et al. Reinforcement learning in multidimensional environments relies on attention mechanisms. J Neurosci 35, 8145–8157 (2015).Balcarras, M., Ardid, S., Kaping, D., Everling, S. & Womelsdorf, T. Attentional Selection Can Be Predicted by Reinforcement Learning of Task-relevant Stimulus Features Weighted by Value-independent Stickiness. J Cogn Neurosci 28, 333–349 (2016).Redish, A. D., Jensen, S., Johnson, A. & Kurth-Nelson, Z. Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. Psychol Rev 114, 784–805 (2007).Nassar, M. R. et al. Rational regulation of learning dynamics by pupil-linked arousal systems. Nat Neurosci 15, 1040–1046 (2012).O’Reilly, J. X. et al. Dissociable effects of surprise and model update in parietal and anterior cingulate cortex. Proc Natl Acad Sci USA 110, 3660–3669 (2013).Shenhav, A., Botvinick, M. M. & Cohen, J. D. The expected value of control: an integrative theory of anterior cingulate cortex function. Neuron 79, 217–240 (2013).Womelsdorf, T. & Everling, S. Long-Range Attention Networks: Circuit Motifs Underlying Endogenously Controlled Stimulus Selection. Trends Neurosci 38, 682–700 (2015).Yang, Y. et al. Nicotinic alpha7 receptors enhance NMDA cognitive circuits in dorsolateral prefrontal cortex. Proc Natl Acad Sci USA 110, 12078–12083 (2013).Aston-Jones, G., Rajkowski, J. & Cohen, J. Role of locus coeruleus in attention and behavioral flexibility. Biological psychiatry 46, 1309–1320 (1999).Cole, B. J. & Robbins, T. W. Forebrain norepinephrine: role in controlled information processing in the rat. Neuropsychopharmacology 7, 129–142 (1992).Dalley, J. W., Cardinal, R. N. & Robbins, T. W. Prefrontal executive and cognitive functions in rodents: neural and neurochemical substrates. Neuroscience and biobehavioral reviews 28, 771–784 (2004).Devauges, V. & Sara, S. J. Activation of the noradrenergic system facilitates an attentional shift in the rat. Behav Brain Res 39, 19–28 (1990).Connor, D. F., Arnsten, A. F., Pearson, G. S. & Greco, G. F. Guanfacine extended release for the treatment of attention-deficit/hyperactivity disorder in children and adolescents. Expert opinion on pharmacotherapy 15, 1601–1610 (2014).Sallee, F. R. et al. Guanfacine extended release in children and adolescents with attention-deficit/hyperactivity disorder: a placebo-controlled trial. J Am Acad Child Adolesc Psychiatry 48, 155–165 (2009).Steere, J. C. & Arnsten, A. F. The alpha-2A noradrenergic receptor agonist guanfacine improves visual object discrimination reversal performance in aged rhesus monkeys. Behav Neurosci 111, 883–891 (1997).Doya, K. Modulators of decision making. Nat Neurosci 11, 410–416 (2008).Wang, X. J. & Krystal, J. H. Computational psychiatry. Neuron 84, 638–654 (2014).Wiecki, T. V. et al. A Computational Cognitive Biomarker for Early-Stage Huntington’s Disease. PLoS One 11, e0148409, doi: 10.1371/journal.pone.0148409 (2016).Huys, Q. J., Pizzagalli, D. A., Bogdan, R. & Dayan, P. Mapping anhedonia onto reinforcement learning: a behavioural meta-analysis. Biol Mood Anxiety Disord 3, 12 (2013).Gershman, S. J. & Niv, Y. Learning latent structure: carving nature at its joints. Curr Opin Neurobiol 20, 251–256 (2010).Voon, V. et al. Disorders of compulsivity: a common bias towards learning habits. Mol Psychiatry 20, 345–352 (2015).Maia, T. V. & Frank, M. J. From reinforcement learning models to psychiatric and neurological disorders. Nature Neuroscience 14, 154–162 (2011).Adams, R. A., Huys, Q. J. M. & Roiser, J. P. Computational Psychiatry: towards a mathematically informed understanding of mental illness. Journal of Neurology, Neurosurgery, and Psychiatry 87, 53–63 (2015).Schlagenhauf, F. et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. NeuroImage 89, 171–180 (2014).Harlé, K. M. et al. Bayesian neural adjustment of inhibitory control predicts emergence of problem stimulant use. Brain 138, 3413–3426 (2015).Zhang, J. et al. Different decision deficits impair response inhibition in progressive supranuclear palsy and Parkinson’s disease. Brain 139, 161–173 (2016).Frank, M. J. et al. fMRI and EEG Predictors of Dynamic Decision Parameters during Human Reinforcement Learning. Journal of Neuroscience 35, 485–494 (2015).Smith, A. C. & Brown, E. N. Estimating a state-space model from point process observations. Neural Comput 15, 965–991 (2003).Wilson, R. C. & Niv, Y. Inferring relevance in a changing world. Frontiers in human neuroscience 5, 189 (2011).Rämä, P. et al. Medetomidine, atipamezole, and guanfacine in delayed response performance of aged monkeys. Pharmacology Biochemistry and Behavior 55, 415–422 (1996).Arnsten, A. F. T. & Contant, T. A. Alpha-2 adrenergic agonists decrease distractibility in aged monkeys performing the delayed response task. Psychopharmacology 108, 159–169 (1992).O’Neill, J. et al. Effects of guanfacine on three forms of distraction in the aging macaque. Life Sciences 67, 877–885 (2000).Wang, M., Ji, J.-Z. & Li, B.-M. The α2A-Adrenergic Agonist Guanfacine Improves Visuomotor Associative Learning in Monkeys. Neuropsychopharmacology 29, 86–92 (2004).Witte, E. a. & Marrocco, R. T. Alteration of brain noradrenergic activity in rhesus monkeys affects the alerting component of covert orienting. Psychopharmacology 132, 315–323 (1997).Decamp, E., Clark, K. & Schneider, J. S. Effects of the alpha-2 adrenoceptor agonist guanfacine on attention and working memory in aged non-human primates. European Journal of Neuroscience 34, 1018–1022 (2011)

Assimilating Seizure Dynamics

Observability of a dynamical system requires an understanding of its state—the collective values of its variables. However, existing techniques are too limited to measure all but a small fraction of the physical variables and parameters of neuronal networks. We constructed models of the biophysical properties of neuronal membrane, synaptic, and microenvironment dynamics, and incorporated them into a model-based predictor-controller framework from modern control theory. We demonstrate that it is now possible to meaningfully estimate the dynamics of small neuronal networks using as few as a single measured variable. Specifically, we assimilate noisy membrane potential measurements from individual hippocampal neurons to reconstruct the dynamics of networks of these cells, their extracellular microenvironment, and the activities of different neuronal types during seizures. We use reconstruction to account for unmeasured parts of the neuronal system, relating micro-domain metabolic processes to cellular excitability, and validate the reconstruction of cellular dynamical interactions against actual measurements. Data assimilation, the fusing of measurement with computational models, has significant potential to improve the way we observe and understand brain dynamics

Prioritized memory access explains planning and hippocampal replay.

To make decisions, animals must evaluate candidate choices by accessing memories of relevant experiences. Yet little is known about which experiences are considered or ignored during deliberation, which ultimately governs choice. We propose a normative theory predicting which memories should be accessed at each moment to optimize future decisions. Using nonlocal 'replay' of spatial locations in hippocampus as a window into memory access, we simulate a spatial navigation task in which an agent accesses memories of locations sequentially, ordered by utility: how much extra reward would be earned due to better choices. This prioritization balances two desiderata: the need to evaluate imminent choices versus the gain from propagating newly encountered information to preceding locations. Our theory offers a simple explanation for numerous findings about place cells; unifies seemingly disparate proposed functions of replay including planning, learning, and consolidation; and posits a mechanism whose dysfunction may underlie pathologies like rumination and craving

An effect of serotonergic stimulation on learning rates for rewards apparent after long intertrial intervals

Serotonin has widespread, but computationally obscure, modulatory effects on learning and cognition. Here, we studied the impact of optogenetic stimulation of dorsal raphe serotonin neurons in mice performing a non-stationary, reward-driven decision-making task. Animals showed two distinct choice strategies. Choices after short inter-trial-intervals (ITIs) depended only on the last trial outcome and followed a win-stay-lose-switch pattern. In contrast, choices after long ITIs reflected outcome history over multiple trials, as described by reinforcement learning models. We found that optogenetic stimulation during a trial significantly boosted the rate of learning that occurred due to the outcome of that trial, but these effects were only exhibited on choices after long ITIs. This suggests that serotonin neurons modulate reinforcement learning rates, and that this influence is masked by alternate, unaffected, decision mechanisms. These results provide insight into the role of serotonin in treating psychiatric disorders, particularly its modulation of neural plasticity and learning.info:eu-repo/semantics/publishedVersio

Hippocampal and prefrontal processing of network topology to simulate the future

Topological networks lie at the heart of our cities and social milieu. However, it remains unclear how and when the brain processes topological structures to guide future behaviour during everyday life. Using fMRI in humans and a simulation of London (UK), here we show that, specifically when new streets are entered during navigation of the city, right posterior hippocampal activity indexes the change in the number of local topological connections available for future travel and right anterior hippocampal activity reflects global properties of the street entered. When forced detours require re-planning of the route to the goal, bilateral inferior lateral prefrontal activity scales with the planning demands of a breadth-first search of future paths. These results help shape models of how hippocampal and prefrontal regions support navigation, planning and future simulation