High-Throughput Operant Conditioning in Drosophila Larvae

Operant conditioning is the process by which animals learn to associate their own behaviour with positive or negative outcomes, biasing future action selection in order to maximise reward and avoid punishment. It is an important strategy to ensure survival in an ever-changing environment. Although operant conditioning has been observed across vertebrate and invertebrate species, the underlying neural mechanisms are still not fully understood. The Drosophila larva is an excellent model system to study neural circuits, since it is genetically tractable, with a variety of tools available. Although it is quite small, it is capable of a diverse range of behaviours and can achieve complex learning tasks. However, while the mechanisms underlying classical conditioning, where animals learn about the appetitive or aversive qualities of an external sensory cue, have been extensively studied in larvae, it has remained an open question whether they are capable of operant conditioning. This is in part due to the challenges which arise during the training process: in order to train an animal to associate its own actions with their outcomes, the experimenter needs to be able to deliver rewarding or punishing stimuli directly in response to behaviour. In this thesis, I introduce a novel high-throughput tracker suitable for training up to 16 larvae simultaneously. I have developed a customised software for real-time detection of various actions that larvae perform: left and right bend, forward crawl, roll and back-up. Light and heat stimuli can be administered at individual animals with minimal delay, enabling optogenetic or thermogenetic activation of circuits encoding reward or punishment in response to behaviour. Using this system, I show that Drosophila larvae are capable of operant conditioning. Pairing bends to one direction, e.g. the left, with optogenetic activation of a large group of reward-encoding dopaminergic and serotonergic neurons is sufficient to induce a learned preference for bending towards this side after training. I explore whether there are other types of actions which larvae can learn to associate with valence, and introduce a second operant conditioning paradigm, in which larvae modify their behaviour following pairing of the stimulus with forward crawls. To identify new candidate neurons signalling valence in a learning context, I also conduct a classical conditioning screen, in which I pair an odour with optogenetic activation of distinct neuron types covered by different driver lines. While activation of many types of gustatory sensory neurons paired with the odour was insufficient for memory formation, I find that the serotonergic neurons of the brain and the subesophageal zone (SEZ) can induce strong appetitive learning. Finally, I show that activity of serotonergic rather than dopaminergic neurons is sufficient for memory formation in the operant bend direction paradigm, and that operant conditioning is impaired when restricting activation to the serotonergic neurons of the brain and the SEZ. My results suggest a novel role of serotonergic neurons for learning in insects as well as the existence of learning circuits outside of the mushroom body. Different subsets of serotonergic neurons mediate classical and operant conditioning. This works lays a foundation for future studies of the function of serotonin and the mechanisms underlying operant conditioning at both circuit level and cellular level.Gates Cambridge Scholarshi

The neuronal substrates of reinforcement and punishment in Drosophila melanogaster

Actions are followed by consequences, to which values are assigned. These subjective values shape our future actions in what we colloquially call “learning by doing”. But how does the assignment of values to behavioral consequences occur in the brain? In mammals as well as in three different superphyla within protostomes (Nematoda, Platyhelminthes and Mollusca) this is mediated by dopamine. However, little is known about the neural basis of value assignment to behaviors in the Arthropoda. In order to address which neurons convey punishment and reinforcement in insects, we performed four different experiments in which flies (Drosophila melanogaster) could control the on/off state of different subsets of dopaminergic neurons. We found that the effects of these neurons across operant (feedback) conditioning have no relation to their role observed in previously well studied pavlovian (feedforward) conditioning. These results suggest fundamentally different neuronal circuits dedicated to operant and pavlovian learning processes. The reinforcing value of most of the tested neurons is context dependent and differs among the tested operant paradigms. However, there are two exceptions: two different cell clusters projecting to different neuropils in the central complex (CX) and accessory regions, a brain area involved in multisensory integration and action selection, seem to be responsible for the reinforcement of motor commands in a context-independent way. These findings support a conserved mechanism of dopaminergic reinforcement in higher order motor centers across phyla

How many neurons are sufficient for perception of cortical activity?

Many theories of brain function assume that information is encoded and behaviour is controlled through sparse, distributed patterns of activity. It is therefore crucial to place a lower bound on the amount of neural activity that can drive behaviour and to understand how neuronal networks operate within these constraints. We use an all-optical approach to test this lower limit by driving behaviour with targeted two-photon optogenetic activation of small ensembles of L2/3 pyramidal neurons in mouse barrel cortex while using two-photon calcium imaging to record the impact on the local network. By precisely titrating the number of neurons in activated ensembles we demonstrate that the lower bound for detection of cortical activity is ~14 pyramidal neurons. We show that there is a very steep sigmoidal relationship between the number of activated neurons and behavioural output, saturating at only ~37 neurons, and that this relationship can shift with learning. By simultaneously measuring activity in the local network, we show that the activation of stimulated ensembles is balanced by the suppression of neighbouring neurons. This surprising behavioural sensitivity in the face of potent network suppression supports the sparse coding hypothesis and suggests that perception of cortical activity balances a trade-off between minimizing the impact of noise while efficiently detecting relevant signals

Two-photon all-optical interrogation of mouse barrel cortex during sensory discrimination

The neocortex supports a rich repertoire of cognitive and behavioural functions, yet the rules, or neural ‘codes’, that determine how patterns of cortical activity drive perceptual processes remain enigmatic. Experimental neuroscientists study these codes through measuring and manipulating neuronal activity in awake behaving subjects, which allows links to be identified between patterns of neural activity and ongoing behaviour functions. In this thesis, I detail the application of novel optical techniques for simultaneously recording and manipulating neurons with cellular resolution to examine how tactile signals are processed in sparse neuronal ensembles in mouse somatosensory ‘barrel’ cortex. To do this, I designed a whisker-based perceptual decision-making task for head-fixed mice, that allows precise control over sensory input and interpretable readout of perceptual choice. Through several complementary experimental approaches, I show that task performance is exquisitely coupled to barrel cortical activity. Using two- photon calcium imaging to simultaneously record from populations of barrel cortex neurons, I demonstrate that different subpopulations of neurons in layer 2/3 (L2/3) show selectivity for contralateral and ipsilateral whisker input during behaviour. To directly test whether these stimulus-tuned groups of neurons differentially impact perceptual decision-making I performed patterned photostimulation experiments to selectively activate these functionally defined sets of neurons and assessed the resulting impact on behaviour and the local cortical network in layer 2/3. In contrast with the expected results, stimulation of sensory-coding neurons appeared to have little perceptual impact on task performance. However, activation of non- stimulus coding neurons did drive decision biases. These results challenge the conventional view that strongly sensory responsive neurons carry more perceptual weight than non-responsive sensory neurons during perceptual decision-making. Furthermore, patterned photostimulation revealed and imposed potent surround suppression in L2/3, which points to strong lateral inhibition playing a dominant role in shaping spatiotemporally sparse activity patterns. These results showcase the utility of combined patterned photostimulation methods and population calcium imaging for revealing and testing neural circuit function during sensorimotor behaviour and provide new perspectives on sensory coding in barrel cortex

행동적 항상성 조절의 신경회로: 섭식 행동과 체온조절 행동 중심으로

학위논문(박사) -- 서울대학교대학원 : 자연과학대학 협동과정 뇌과학전공, 2022. 8. 김성연.Maintaining physiological conditions (e.g., nutrients, water, and body temperature) within a narrow range is a distinct quality of living organisms. Among them, animals utilize diverse behaviors to suffice materials in need to maintain internal stability, called homeostasis. As such, understanding the neural mechanisms for behavioral regulations of homeostatic conditions has been a key interest in neurobiology and physiology. However, despite the importance of this question, neural processes underlying some of the fundamental behaviors for maintaining organismal homeostasis are still elusive. In this dissertation, I describe the data revealing neural circuit mechanisms underlying two basic behaviors for regulating homeostasis: ingestive and thermoregulatory behaviors. Chapter one of this dissertation briefly overviews the origin of the homeostasis concept and provides a short historical perspective on studying neural processes for behavioral regulation of homeostasis. Chapter two describes a molecularly defined neural population in a hindbrain area called the parabrachial nucleus for monitoring and suppressing ingestion and demonstrates a gut-to-brain neural circuit spanning from the peripheral sensory ganglia to the hypothalamus for mechanosensory feedback control of ingestion. Chapter three describes a parabrachial-to-hypothalamic neural circuit for thermoregulatory behaviors and the neural coding of motivational aspects of thermal stimuli by a subpopulation of hypothalamic neurons. Chapter four describes a summary of the data presented in this dissertation and provides directions for further investigations. Together, this dissertation presents studies on understanding neural circuit mechanisms underlying fundamental behaviors for regulating crucial homeostatic conditions, including energy, fluid, and body temperature.영양분과 체온 등 체내 환경을 일정 수준으로 유지하는 성질인 항상성은 생명체의 중요한 특징이다. 여러 생물 종 중에서 특별히 동물들은 행동을 통해 자신의 필요를 채워 체내 항상성을 유지하는데, 따라서 항상성을 유지하기 위한 동물 행동을 매개하는 신경 기작을 이해하는 것은 신경생물학 및 생리학의 오랜 과제였다. 수십 년에 걸친 연구들로 많은 부분이 밝혀졌으나, 몇몇 근본적인 항상성 유지 행동을 매개하는 신경회로 기작은 여전히 알려지지 않고 있었다. 본 연구는 섭식 행동과 체온 유지 행동을 조절하는 데 관여하는 새로운 신경회로 기작을 밝힘으로써, 항상성 유지의 신경학적 원리에 대한 이해를 넓히는 것을 목표로 한다. 1장에서는 항상성 개념의 형성과 그 이후 이어진 항상성 조절 신경 기작에 관한 연구의 역사를 간략히 살펴보고자 한다. 2장에서는 소화관의 물리적 감각 신호를 이용하여 섭식을 모니터링하고 이에 따라 음식물의 섭취를 조절하는 후뇌부 신경세포 집단과, 그 집단을 중심으로 말초 신경절에서 시상하부까지 이어지는 장-뇌 신경회로에 관한 연구를 기술한다. 3장에서는 다양한 체온조절 행동에 필수적인 시상하부 신경 집단과 이를 중심으로 체온 조절 행동을 매개하는 신경회로 기작에 관한 연구를 기술한다. 4장에서는 본 논문에 기술된 연구 결과를 간략히 정리하고 그 의의에 관해 살펴본다. 본 연구들은 섭식 행동과 체온 조절 행동을 매개하는 신경회로 기작에 관한 새로운 이해를 더함으로써, 생명 유지에 필수적인 항상성 조절 행동의 신경학적 기반을 완전히 밝히는 데 기여할 것으로 기대한다.Chapter 1. Introduction 1 Chapter 2. A neural circuit mechanism for mechanosensory feedback control of ingestion 6 Chapter 3. A forebrain neural substrate for behavioral thermoregulation 80 Chapter 4. Conclusion 167 Bibliography 173 Abstract in Korean 186박

Role of dopamine tone in brain stimulation reward

The experiments described in the present thesis address the specific role of dopamine (DA) tone in brain stimulation reward (BSR). The level of extracellular dopamine in the rat nucleus accumbens was measured by means of in-vivo microdialysis in rats receiving electrical stimulation of the medial forebrain bundle. The first experiments characterize changes in DA tone as a function of reward predictability and duty cycle under circumstances in which phasic release of DA, as measured by fast-scan cyclic voltammetry, has been reported to be absent (Garris et al., 1999). The results obtained using several different reinforcement schedules suggest that DA tone reflects the duty cycle of the stimulation rather than the predictability of the reward. In contrast to the transient elevation observed when stimulation is delivered at short (1.5 s) inter-train intervals, stimulation trains separated by long (12 s) inter-train intervals can sustain a stable level of DA tone for up to two hours. This difference in the stability of DA tone has repercussions for the behaviour sustained by BSR, as measured by means of the curve-shift paradigm. When DA tone was measured under similar circumstances to those in the Garris et al. (1999) study, a robust increase in tonic DA release was observed, on each schedule tested, in sharp contrast to the transient changes in phasic DA release described by Garris et al. (1999). These results suggest that phasic and tonic DA release are under differential control. In an additional experiment, the reinforcement-mountain model and testing paradigm were used to determine the stage(s) of processing at which DA tone influences the pursuit of BSR. The mountain model relates pursuit of BSR to the cost and strength of the electrical stimulation. DA reuptake was blocked by continuous subcutaneous infusion of cocaine in a novel manner that avoids tissue damage. The 3D structure defined by time-allocation, reward cost and reward strength. always shifted rightward along the cost axis but rarely along the strength axis. This result implies that the leftward shifts seen in "rate-frequency" studies of the effect of cocaine on intracranial self-stimulation (ICSS) are misleading: these effects are due to displacement of the diagonally oriented face of the 3D structure along the cost axis. The results demonstrate that DA tone exerts its influence at a later processing stage than originally proposed and long believed

Behavior is movement only but how to interpret it? Problems and pitfalls in translational neuroscience-a 40-year experience

Translational research in behavioral neuroscience seeks causes and remedies for human mental health problems in animals, following leads imposed by clinical research in psychiatry. This endeavor faces several problems because scientists must read and interpret animal movements to represent human perceptions, mood, and memory processes. Yet, it is still not known how mammalian brains bundle all these processes into a highly compressed motor output in the brain stem and spinal cord, but without that knowledge, translational research remains aimless. Based on some four decades of experience in the field, the article identifies sources of interpretation problems and illustrates typical translational pitfalls. (1) The sensory world of mice is different. Smell, hearing, and tactile whisker sensations dominate in rodents, while visual input is comparatively small. In humans, the relations are reversed. (2) Mouse and human brains are equated inappropriately: the association cortex makes up a large portion of the human neocortex, while it is relatively small in rodents. The predominant associative cortex in rodents is the hippocampus itself, orchestrating chiefly inputs from secondary sensorimotor areas and generating species-typical motor patterns that are not easily reconciled with putative human hippocampal functions. (3) Translational interpretation of studies of memory or emotionality often neglects the ecology of mice, an extremely small species surviving by freezing or flight reactions that do not need much cognitive processing. (4) Further misinterpretations arise from confounding neuronal properties with system properties, and from rigid mechanistic thinking unaware that many experimentally induced changes in the brain do partially reflect unpredictable compensatory plasticity. (5) Based on observing hippocampal lesion effects in mice indoors and outdoors, the article offers a simplistic general model of hippocampal functions in relation to hypothalamic input and output, placing hypothalamus and the supraspinal motor system at the top of a cerebral hierarchy. (6) Many translational problems could be avoided by inclusion of simple species-typical behaviors as end-points comparable to human cognitive or executive processing, and to rely more on artificial intelligence for recognizing patterns not classifiable by traditional psychological concepts

Investigations into the role of the metabotropic glutamate receptor, mGluR5, in incentive learning and some behavioural and neurobiological effects of cocaine

The metabotropic glutamate receptor, mGluR5, is densely expressed in brain regions involved in incentive learning processes. There is considerable evidence to suggest that following exposure to addictive drugs such as cocaine, adaptations in these brain areas may underlie the development and maintenance of behavioural responses related to addictive processes. The present thesis examines the role of mGluR5 in both incentive learning processes and some behavioural and neurobiological effects of cocaine. First, using a novel mutant mouse line in which mGluR5 is selectively knocked down in cells that express dopamine D1 receptors (D1R), I argue that this mGluR5 population is critically important for specific incentive learning processes. By blocking mGluR5 in wild-type mice with a selective antagonist, I then propose mGluR5 as necessary for the acquisition, but not the expression of an incentive association. Next, I present data showing that mGluR5 on dopaminoceptive neurons are not necessary for the „conditioned rewarding‟ properties of cocaine, measured in the conditioned place preference model, but do contribute to the psychomotor activating effects of cocaine. Finally, I present an immunohistochemistry study that examines cocaine-induced activation of the extracellular-signal related kinase (ERK) pathway. In the mGluR5 knock-down mice, activation of the ERK pathway in the striatum is disrupted following an acute injection of cocaine. Given the importance of the ERK pathway in establishing and maintaining long term memories, I propose that disruption of this pathway could contribute, in part, to some findings reported in the present thesis. Taken together, this thesis will argue that signalling through mGluR5 on D1R expressing neurons is important for the formation of incentive associations, and may contribute to neural adaptations necessary for the development and maintenance of behavioural responses related to addictive processes