Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning

Navigation is a complex skill with a long history of research in animals and humans. In this work, we simulate the Morris Water Maze in 2D to train deep reinforcement learning agents. We perform automatic classification of navigation strategies, analyze the distribution of strategies used by artificial agents, and compare them with experimental data to show similar learning dynamics as those seen in humans and rodents. We develop environment-specific auxiliary tasks and examine factors affecting their usefulness. We suggest that the most beneficial tasks are potentially more biologically feasible for real agents to use. Lastly, we explore the development of internal representations in the activations of artificial agent neural networks. These representations resemble place cells and head-direction cells found in mouse brains, and their presence has correlation to the navigation strategies that artificial agents employ.Comment: 26 pages, 15 figure

Odor interactions and learning in a model of the insect antennal lobe

Abstract We present a new model of insect antennal lobe in the form of integro-di erential equation with short-range inhibition. The learning in the model modiÿes the odor-dependent input by adding a term that is proportional to the ÿring rates of the network in the pre-learning steady state. We study the modiÿcation of odor-induced spatial patterns (steady states) by combination of odors (binary mixtures) and learning. We show that this type of learning applied to the inhibitory network "increases the contrast" of the network's spatial activity patterns. We identify pattern modiÿcations that could underlie insect behavioral phenomena

Investigating the ability of astrocytes to drive neural network synchrony

Recent experimental works have implicated astrocytes as a significant cell type underlying several neuronal processes in the mammalian brain, from encoding sensory information to neurological disorders. Despite this progress, it is still unclear how astrocytes are communicating with and driving their neuronal neighbors. While previous computational modeling works have helped propose mechanisms responsible for driving these interactions, they have primarily focused on interactions at the synaptic level, with microscale models of calcium dynamics and neurotransmitter diffusion. Since it is computationally infeasible to include the intricate microscale details in a network-scale model, little computational work has been done to understand how astrocytes may be influencing spiking patterns and synchronization of large networks. We overcome this issue by first developing an “effective” astrocyte that can be easily implemented to already established network frameworks. We do this by showing that the astrocyte proximity to a synapse makes synaptic transmission faster, weaker, and less reliable. Thus, our “effective” astrocytes can be incorporated by considering heterogeneous synaptic time constants, which are parametrized only by the degree of astrocytic proximity at that synapse. We then apply our framework to large networks of exponential integrate-and-fire neurons with various spatial structures. Depending on key parameters, such as the number of synapses ensheathed and the strength of this ensheathment, we show that astrocytes can push the network to a synchronous state and exhibit spatially correlated patterns

Diversity of Evoked Astrocyte Ca2+ Dynamics Quantified through Experimental Measurements and Mathematical Modeling

Astrocytes are a major cell type in the mammalian brain. They are not electrically excitable, but generate prominent Ca2+ signals related to a wide variety of critical functions. The mechanisms driving these Ca2+ events remain incompletely understood. In this study, we integrate Ca2+ imaging, quantitative data analysis, and mechanistic computational modeling to study the spatial and temporal heterogeneity of cortical astrocyte Ca2+ transients evoked by focal application of ATP in mouse brain slices. Based on experimental results, we tune a single-compartment mathematical model of IP3-dependent Ca2+ responses in astrocytes and use that model to study response heterogeneity. Using information from the experimental data and the underlying bifurcation structure of our mathematical model, we categorize all astrocyte Ca2+ responses into four general types based on their temporal characteristics: Single-Peak, Multi-Peak, Plateau, and Long-Lasting responses. We find that the distribution of experimentally-recorded response types depends on the location within an astrocyte, with somatic responses dominated by Single-Peak (SP) responses and large and small processes generating more Multi-Peak responses. On the other hand, response kinetics differ more between cells and trials than with location within a given astrocyte. We use the computational model to elucidate possible sources of Ca2+ response variability: (1) temporal dynamics of IP3, and (2) relative flux rates through Ca2+ channels and pumps. Our model also predicts the effects of blocking Ca2+ channels/pumps; for example, blocking store-operated Ca2+ (SOC) channels in the model eliminates Plateau and Long-Lasting responses (consistent with previous experimental observations). Finally, we propose that observed differences in response type distributions between astrocyte somas and processes can be attributed to systematic differences in IP3 rise durations and Ca2+ flux rates

Receptor recharge time drastically reduces the number of captured particles

<div><p>Many diverse biological systems are described by randomly moving particles that can be captured by traps in their environment. Examples include neurotransmitters diffusing in the synaptic cleft before binding to receptors and prey roaming an environment before capture by predators. In most cases, the traps cannot capture particles continuously. Rather, each trap must wait a transitory “recharge” time after capturing a particle before additional captures. This recharge time is often overlooked. In the case of instant recharge, the average number of particles captured before they escape grows linearly in the total number of particles. In stark contrast, we prove that for any nonzero recharge time, the average number of captured particles grows at most logarithmically in the total particle number. This is a fundamental effect of recharge, as it holds under very general assumptions on particle motion and spatial domain. Furthermore, we characterize the parameter regime in which a given recharge time will dramatically affect a system, allowing researchers to easily verify if they need to account for recharge in their specific system. Finally, we consider a few examples, including a neural system in which recharge reduces neurotransmitter bindings by several orders of magnitude.</p></div