We examine a simple model of mammalian visual system. This structure is simulated by means of several hundred Hodgkin-Huxley neurons. We investigate signal processing properties of the model. Some methods taken from informational theory are applied to the analysis of Primary Visual Cortex' dynamics. Discussion of efficiency of such methods in two dimensional movement detection is presented in some detail

A. Kamiński, Wiesław

M. Wójcik, Grzegorz

English

University of Maria Curie-Skłodowska (UMCS): Scientific e-Journals / Uniwersytet Marii Curie-Skłodowskiej: e-czasopisma naukowe

 Annales UMCS Informatica AI 3 (2005) 145-152 Annales UMCS Informatica Lublin-Polonia  Sectio AI http://www.annales.umcs.lublin.pl/  Investigating Mammalian Visual System with methods  of informational theory  Grzegorz M. Wójcik∗, Wiesław A. Kamiński  Institute of Computer Science, Maria Curie-Skłodowska University,  pl. Marii Curie-Skłodowskiej 1, 20-031 Lublin, Poland  Abstract We examine a simple model of mammalian visual system. This structure is simulated by means of several hundred Hodgkin-Huxley neurons. We investigate signal processing properties of the model. Some methods taken from informational theory are applied to the analysis of Primary Visual Cortex’ dynamics. Discussion of efficiency of such methods in two dimensional movement detection  is presented in some detail.  1. Introduction Idea of cortical microcircuits simulations was suggested by Maass and since then it has been called Liquid State Machine (LSM) [1]. In general, the fragment of neocortex is treated as a liquid. It was proved that such structures may have great computational abilities [2]. In this paper we simulate a simple mammalian visual system based on the LSM model [1]. However, we will not analyse its computational abilities here. Instead of this, we will concentrate on investigating the dynamics of Primary Visual Cortex (PVC) and its investigation with some methods originating from the informational theory [3]. Mammalian brains are built of neurons organised in microcircuits [4]. Microcircuits form columns, and the function of each column depends on its location in the brain. In our model we create 4-neuron microcircuits. An input device (ID) representing the retina is built of 25 microcircuits. As a part of primary visual cortex and Lateral Geniculate Nucleus (LGN) – the liquid – we build a set of 25 Hodgkin-Huxley1 Liquid State Machines (HHLSMs). There are 24 neurons in each HHLSM. Randomly chosen microcircuits of ID are connected with randomly chosen HHLSMs. The same architecture of connections is arranged                                                  ∗Corresponding author: e-mail address: gmwojcik@hektor.umcs.lublin.pl 1Neurons are built according to Hodgkin-Huxley model [5]. Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSGrzegorz M. Wójcik, Wiesław A. Kamiński 146between the cortex and each of two “readout” devices (readout A and readout B), both consisting of 100 neurons (for detail see. Fig. 1). Readouts play a role of PVC parts and in this case they are specialised in movement detection. To achieve this, we arrange additionally some special architecture of connections inside each readout. Weight values of these connections increase exponentially in horizontal (in readout A) or vertical direction (in readout B).  The value of each weight is estimated according to the following rule:  5%ew i/4 ±= , (1) where i is the coordinate of readout vertical (readout A) or horizontal (readout B) row of cells and it varies from 1 to 10. Readout neurons in each vertical or horizontal row are full-connected without auto-connections. Note that the values of weights are randomly diffused according to (1) with the accuracy of ±5%. Fig. 2 presents typical maps of readout weights. One can choose some other architectures of readout connections, i.e. with weight values increasing linearly. There are also arranged some random connections among HHLSMs, however, their number, as in real PVC, is not very large2.   Fig. 1. Scheme of simulated biological-like visual system  All of the structures described above were simulated in the GEneral NEural SImulation System (GENESIS) [6].  The neurons of ID are then stimulated by a series of four different moving patterns (see Fig. 3). By the “input pattern” here we understand the electric stimulation of  some chosen set of retinal cells. Input patterns are transformed by the liquid. As a result, activity responses of both readouts are collected.                                                    2 HHLSM columns are connected with probability p=5%. Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSInvestigating Mammalian Visual System with methods … 147READOUT A    READOUT B    Fig. 2. Maps of readout weights 3D (left panel) and 2D (right panel)  (a)  (b)  (c)  (d)  Fig. 3. Four different stimulation patterns: bar moving from left to right (a), from right to left (b), from top to bottom (c) and from bottom to the top of ID (d).  Arrows represent the direction of the movement  The aim of this contribution is to show, that the methods known from Shannon’s informational theory [8] allow for effective movement classification and are helpful in complex pattern recognition.  Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSGrzegorz M. Wójcik, Wiesław A. Kamiński 1482. Theory and results For completeness we summarise main features of entropy based on the classical definition of Shannon’s informational entropy [3, 8]. We simulate T=100 ms of biological model activity. Though, N spike potentials on chosen readout are obtained for each pattern shown to ID. Thus, for each readout we can introduce the probability of ni spikes observation. These spikes occurred within a period ti=0.1 ms:  iinpN= . (2) Such probability is then interpreted as a chance of giving the whole information in ith interval of time T. All probabilities for time T sum up to 1:  10001 2 1000ii 1n n ... np 1N=+ + += =∑ . (3) Having probability pi defined we introduce individual entropies  ( )i0 i iS p ln p= − ⋅  (4) and their sum as the global entropy S.  ( )i0 i ii iS S p ln p= = − ⋅∑ ∑ . (5) Global and individual informational entropies for 9 different input patterns were discussed in [7].  Next, using Eqs. (4)-(5) we introduce the individual joined entropy  ( )ij0 ij ijH p ln p= − ⋅  (6) and global joined entropy  ( )ij0 ij iji j i jH H p ln p= = −∑∑ ∑∑ , (7) where probability pij in Eqs. (6-7) is interpreted as a chance of transmission of the whole information characterising  the pattern i in time ti and the pattern j in time tj. Finally, for two different input patterns i, j, we introduce the Mutual Information (MI):  ij i j ij0 0 0 0M S S H= + − . (8) Eqs. (4)-(6) let us obtain the formula for individual MI  ( )( ) ( )( ) ( )ij0 i i j j ij ijM p ln p p ln p p ln p= − ⋅ + − ⋅ − ⋅ . (9) Simple modification of Eq. (9) leads to the global MI definition:  ( ) ( ) ( )ij i i j j ij iji j ijM p ln p p ln p p ln p⎛ ⎞ ⎛ ⎞⎛ ⎞= − ⋅ + − ⋅ − − ⋅⎜ ⎟ ⎜ ⎟⎜ ⎟⎝ ⎠ ⎝ ⎠ ⎝ ⎠∑ ∑ ∑ . (10) MI can be interpreted as a measure of patterns similarity [3]. It is often used in pictures fitting: two pictures are best fitted when the MI reaches its maximum. Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSInvestigating Mammalian Visual System with methods … 149Multidimensional analysis of MI changes for different time intervals was discussed in [9]. In this contribution we will apply the method for investigating moving patterns. Figs. 4-5 show the changes of individual and global MI between two readouts in the case of stimulation moving from left to right in the horizontal direction (see. Fig. 3a). In contrast, Figs. 6-7 present MI changes for the vertical moving stimuli (see Fig. 3d).    Fig. 4. Individual MI (in the left panel) and global MI (in the right panel) between two readout devices for the horizontal pattern (see. Fig 3a). 3D MI maps were generated  for the first 50 ms of simulation. For detail see text    Fig. 5. Individual MI (in the left panel) and global MI (in the right panel) between two readout devices for the horizontal pattern (see. Fig 3a). 2D MI maps were generated  for the first 50 ms of simulation  Although in both cases maps of the global MI are very similar, the individual MI can be applied for recognition of the horizontal or vertical orientation of the movement. For horizontal stimulation (see Fig. 4) we obtain some higher horizontal “belt of peaks”. For the second case (see Fig. 5) peaks are higher for the vertical direction as can be seen in the plot of MI map. However, such attitude does not give any information about  the direction of the movement.  Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSGrzegorz M. Wójcik, Wiesław A. Kamiński 150  Fig. 6. Individual MI (left panel) and global MI (right panel) between two readout devices for the vertical pattern (see. Fig 3d). 3D MI maps were generated for the first 50 ms of simulation.  For detail see text    Fig. 7. Individual MI (left panel) and global MI (right panel) between two readout devices for the vertical pattern (see. Fig 3d). 2D MI maps were generated for the first 50 ms of simulation  To achieve the above mentioned purposes we suggest application of energetic function. In analogy to the informational entropy and MI we can define individual energy i0T :  i0 jklm jk lmj,k l,mT w y y= ⋅ ⋅∑∑  (11) and global energy  i0 jklm jk lmi i j,k l,mT T w y y= = ⋅ ⋅∑ ∑∑∑ , (12) where wjklm is the weight of connection betwen two readout neurons (with coordinates j,k and l,m). Activity yjk (ylm) corresponds to the mean potential value of the neuron j, k (l, m) measured over the time interval ti. Fig. 8 presents changes of energetic function for readout A and horizontal stimulation (Fig. 3a) with different time intervals ti=0.1 ms and ti=5 ms.  Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSInvestigating Mammalian Visual System with methods … 151In analogy changes of energetic function for readout B and vertical pattern (Fig. 3d) are shown in Fig. 9.   0 20 40 60 80 100Time of Simulation [ms]02000400060008000Energetic Function 0 20 40 60 80Time of Simulation [ms]10000100000Energetic Function Fig. 8. Energetic  function for readout A for ti=0.1 ms in the left panel and for ti=5 ms in the right panel for the horizontal pattern (Fig 3a)  0 20 40 60 80 100Time of Simulation [ms]02000400060008000Energetic Function 0 20 40 60 80Time of Simulation [ms]10000100000Energetic Function Fig. 9. Energetic function of readout B for ti=0.1 ms in the left panel and for ti=5 ms in the right panel for vertical pattern (Fig 3d)  The results of the above simulations show that in our model appropriate readouts are specialised for the detection of two kinds of movement. For the case when the energetic function of readout A (or readout B) grows together with the weight growth (as in Fig. 8) we can conclude that ID was stimulated from left to right (or from top to bottom). Respectively, if the energetic function of readout A (or readout B) drops along with the weight growth (as in Fig. 9) one can denote the movement direction from right to left (or from bottom to top) of ID. Application of readout A for the vertical movement detection and readout B for the horizontal one would be useless as can be seen in Fig. 10.   Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCSGrzegorz M. Wójcik, Wiesław A. Kamiński 1520 20 40 60 80 100Time of Simulation [ms]0200040006000800010000Energetic Function 0 20 40 60 80 100Time of Simulation [ms]02000400060008000Energetic Function Fig. 10. Left panel: energetic  function of readout A for ti=0.1 ms and vertical pattern (Fig 3a). Right panel: energetic  function of readout B for ti=0.1 ms and horizontal pattern (Fig 3d).  For detail see text  3. Summary In conclusion, we proved that the methods derived from theory of information can be used in computational neuroscience applications as effective movement classifiers. In future, functions like informational entropy, mutual information, and energetic function may become very helpful in implementation of dynamic pattern recognition abilities. First applications of the methods described in this report will probably be found in the visual systems of intelligent robots.  References [1] Maass W., Natschlaeger T., Markram H.,  Real-time Computing Without Stable States: A New Framework for Neural Computation Based on Perturbation, Neural Computations, 14(11) (2002) 2531. [2]  Holden A.V., Tucker J.V., Thompson B.C., Can excitable media be considered as computational systems?, Physica D, 49 (1991) 240. [3] Cover T., Thomas J., Elements of Information Theory, Wiley, New York. [4] Gupta A., Wang Y., Markram H., Organizing principles for a diversity of GABAergic interneurons and synapses in the neocortex, Science 287 (2000) 273. [5] Hodgkin A., Huxley A., Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligio, J. Physiol., London (1952). [6] Bower J.M., Beeman D., The Book of GENESIS – Exploring Realistic Neural Models with the GEneral NEural SImulation System, Telos, New York (1995). [7] Kamiński W.A., Wójcik G.M., Liquid State Machines Built of Hodgkin-Huxley Neurons – Pattern  Recognition and Informational Entropy, Annales UMCS Informatica, 1, (2003). [8] Shannon C.E., A mathematical theory of communication, Bell Systems Technical Journal, 27 (1948) 379 and 623. [9] Wójcik G.M., Kamiński W.A., Multidimensional Mutual Information in Biological Visual System, Artificial Intelligence, Wydawnictwo Akademii Podlaskiej, Siedlce (2004).   Pobrane z czasopisma Annales AI- Informatica http://ai.annales.umcs.plData: 04/08/2020 20:39:50UMCPowered by TCPDF (www.tcpdf.org)

Investigating Mammalian Visual System with methods of informational theory

https://journals.umcs.pl/ai/article/download/3016/2212

Investigating Mammalian Visual System with methods of informational theory

Abstract

Similar works

Full text

Available Versions

University of Maria Curie-Skłodowska (UMCS): Scientific e-Journals / Uniwersytet Marii Curie-Skłodowskiej: e-czasopisma naukowe