The brain is, without any doubt, the most complex system of the human body. Its complexity is also due to the extremely high number of neurons, as well as the huge number of synapses connecting them. Each neuron is capable to perform complex tasks, like learning and memorizing a large class of patterns. The simulation of large neuronal systems is challenging for both technological and computational reasons, and can open new perspectives for the comprehension of brain functioning. A well-known and widely accepted model of bidirectional synaptic plasticity, the BCM model, is stated by a differential equation approach based on bistability and selectivity properties. We have modified the BCM model extending it from a single-neuron to a whole-network model. This new model is capable to generate interesting network topologies starting from a small number of local parameters, describing the interaction between incoming and outgoing links from each neuron. We have characterized this model in terms of complex network theory, showing how this learning rule can be a support for network generation

Bersani, F.

Castellani, Gastone

Giampieri, E.

Milanesi, L.

Remondini, D.

Verondini, E.

Zironi, I.

English

Scientific Open-access Literature Archive and Repository

DOI 10.1393/ncc/i2009-10396-5Colloquia: CSFI 2008IL NUOVO CIMENTO Vol. 32 C, N. 2 Marzo-Aprile 2009Large-scale modelling of neuronal systemsG. Castellani(1)(∗), E. Verondini(2), E. Giampieri(2), L. Milanesi(3),F. Bersani(2), I. Zironi(3) and D. Remondini(2)(1) DIMORFIPA, Universita` di Bologna and INFN, Sezione di BolognaBologna, Italy(2) Dipartimento di Fisica, Universita` di Bologna and INFN, Sezione di BolognaBologna, Italy(3) ITB-CNR Milano, Italy(ricevuto il 15 Maggio 2009; pubblicato online il 7 Settembre 2009)Summary. — The brain is, without any doubt, the most complex system of thehuman body. Its complexity is also due to the extremely high number of neurons,as well as the huge number of synapses connecting them. Each neuron is capableto perform complex tasks, like learning and memorizing a large class of patterns.The simulation of large neuronal systems is challenging for both technological andcomputational reasons, and can open new perspectives for the comprehension ofbrain functioning. A well-known and widely accepted model of bidirectional synap-tic plasticity, the BCM model, is stated by a differential equation approach basedon bistability and selectivity properties. We have modified the BCM model extend-ing it from a single-neuron to a whole-network model. This new model is capableto generate interesting network topologies starting from a small number of localparameters, describing the interaction between incoming and outgoing links fromeach neuron. We have characterized this model in terms of complex network theory,showing how this learning rule can be a support for network generation.PACS 87.18.Vf – Systems biology.PACS 87.18.Sn – Neural networks and synaptic communication.PACS 87.75.Hc – Networks and genealogical trees.1. – DescriptionWe start from the Bienenstock, Cooper and Munro (BCM) nonlinear model for thesynaptic plasticity of single neuron synapses (fig. 1), which represents each incomingsynapse with a [0; 1] real number, and expresses the time evolution of each incomingsynapse as a function of the neuron output, the values of the synapses and previous(∗) E-mail: gastone.castellani@unibo.itc© Societa` Italiana di Fisica 1314 G. CASTELLANI, E. VERONDINI, E. GIAMPIERI, ETC.Fig. 1. – Scheme of a synaptic junction. Left side (pre-synaptic) is the axon end, right side(post-synaptic) is the dendrite.Fig. 2. – The moving threshold, as stated by the BCM theory.neuron firing history. This formulation determines a high level of selectivity of the neuron,which at the end of the training becomes responsive to only one of the incoming stimuli.We were searching for a simple model describing a neural network evolution based onthe BCM model, which obtained great experimental confirmation for the primary visualcortex development. We simplified BCM equations (generalizing them in some way) andadded to the model few parameters modulating the competition between each group oflinks (e.g. incoming and outgoing). These groups emerge intuitively in the transitionbetween the single neuron and the network model. The classical network evolutionmodels [1], Barabasi-Alberts preferential attachment rule [2], and the Watts-Strogatzrewiring model [3], were discarded because we were looking for a continuous model whichcould determine the dynamics of each link with only local information available.Bienenstock, Cooper and Munro proposed [4] for the first time a model for synapticplasticity which could generate long-term depotentiation (LTD) without external impo-sitions. This model was developed as a heuristic model for explaining the experimentaldata of the sensorial-cortex neurons selectivity to the inputs. It is based on the Hebbianmodel, replacing the Φ constant with a Φ(y, θM ), nonlinear function of y and θM , a pa-rameter that regulates the plasticity direction: LTD if y < θM and long-term potentiationLTP if y > θM .The Φ(y, θM ) function shape is displayed in fig. 2. The competition is incorporated inthe equation defining θM as a function of the squared mean of the output [4]. This forcesthe selection threshold to adapt to the output levels as time goes by. The synapses withan activation level below the neuron activation will be depotentiated, while the synapseswith a stronger activation value will be potentiated, leaving only the latter in the end.LARGE-SCALE MODELLING OF NEURONAL SYSTEMS 15The system is described by the equations below:(1)⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩y = σ(∑iωixi),dwidt= y (y − θM )xiσ′(y),θM = E[y2],in which E[ ] represents a time average and, for large averaging periods, corresponds toan average over the input distribution. From a statistical point of view, these equationsperform a minimization of the following objective function [5]:R =14E[y2]2 − 13E[y3],which quantifies the difference from a Gaussian (symmetric) distribution. The BCMselection rule is therefore equivalent to the input combination which maximizes the non-Gaussian form of the output.This discriminating behavior for input signals has some interesting properties, firstof all the linearity of the objective function gradient with respect to the dimensionalityof the problem, keeping it tractable even in high-dimensional spaces. The selectivityalso allows a research of the projections of the synaptic weight vector orthogonal tothe whole input vectors except one, creating a state completely described by K optimalprojections, instead of 12K(K−1) planes which describe the boundaries between clusters.This property is cardinal when the problem dimensionality is high, because it avoids thecurse of dimensionality, which requires an exponentially growing amount of data withthe dimension increase for sufficient statistics [6].We worked on a set of equations similar to the original BCM model [7], expandingit into a multi-neuronal system under the hypothesis of a slow variation of the synapticweight, so to substitute the time-average with the instantaneous value of each link.Calling xij the weight of the synapses starting from neuron i to neuron j, we consideredthe following general equation:(2) x˙ij = xij⎛⎝xij − xij2 − aijkl ∑i=k,j =lx2kl⎞⎠ ,in which aijkl is a fourth-order tensor which describes the interaction of each pair ofsynapses. A positive value of aijkl corresponds to a competition between link xij andxkl, a negative value to a cooperation. The simplest case is the so-called homogeneouscompetition, with a constant tensor aijkl = a∀i, j, k, l. This case is useful for mathe-matical analysis but is too simplicistic, so we considered local interaction between linkssubdividing them into equivalence classes of competition. Imposing a local interaction(only synapses belonging to the same neurons could interact) we subdivided the tensorinto four main classes of interaction: ingoing links, outgoing links, hierarchical links andreciprocal. The ingoing links are those links that enter into the same neuron, and werethe only ones considered in the BCM model. The outgoing links are those that exit fromthe same neuron, and the competition among them is due to the energy cost of the main-tenance of a lot of synapses. The hierarchical competition is a different selection rule16 G. CASTELLANI, E. VERONDINI, E. GIAMPIERI, ETC.that forces the neuron to sacrifice the incoming or the outgoing connection, specializinginto an input or output neuron. The last one, the reciprocal competition, controls thereciprocal connections between neurons.dxijdt= x2ij − xij⎡⎣x2ij + Arx2ji + Ai∑l =jx2il + Ao∑k =ix2kj + Ag⎛⎝∑k =jx2ki +∑l =ix2jl⎞⎠⎤⎦.(3)2. – AnalysisGeneric model analysis. – First we started with the analysis of the model of N nodeswith symmetrical and homogeneous competition. The evolution equation can be treatedanalytically when there are only two nodes: when the modulating parameter A goesbelow a critical value, the competition is weak enough to allow both links to survive,and we observe a phase transition even in this simple bidimensional system. Then weextended these results to the n-dimensional system, showing that the system always hasa Lyapunov function if the interaction between nodes is symmetrical, and the systemshows an exponential convergence to a stable state with at maximum A− 1 alive links,distributed approximately with a Poisson function. This interaction topology can beinterpreted as a mean-field interaction.Network topological proximity . – We restricted the concept of mean field from thewhole network to a subset of links, chosen from the topological proximity of each linkwith respect to their starting and ending nodes. We define a competition group consistingof all links incoming to the same node, one for all outgoing links and another, calledhierarchical competition, comprising the incoming and outgoing links of the same node.So we have now three different parameters independent from the system dimensionality.These values are positive when two links are in reciprocal competition, and negativewhen they are collaborating. This model analysis is far over the pen and paper systempossibility, so we worked on a wide number of simulations. When the system is composedof N nodes and N(N−1) links, the ideal algorithmic convergence of this model is O(N2),but with some simple numerical approximation we can work at O(NB) with B a realnumber in the interval [1; 2] with strong parameter and initial condition dependencies.Bow-tie structure. – A strong hierarchical competition forces a node to choose betweenincoming and outgoing links. Loosening this condition we can allow the node to stabilizein a intermediate status, with few incoming and outgoing links. The global structureis similar to the hypothesized properties of a human-made network, the World WideWeb [8], which is divided into three groups: the exit pages, with only outgoing links, theentering pages, with only incoming links, and an intermediate, strongly interconnectedgroup with both incoming and outgoing links (see fig. 3).Metabolic pathways structure. – This network represents the interaction of variousproteins involved in metabolic functions of the cell. Each protein has a really specificinteraction target, so we can impose that only one, two only in few cases, outgoing linksare allowed for each protein (typical outgoing competition parameter of 0.3). We cansee in fig. 4 that the resulting network has the same features as a metabolic network. Abetter approximation is possible by introducing a small collaboration between incominglinks (co-regulation is a very common feature, so we can hypothesize to be present inthis case).LARGE-SCALE MODELLING OF NEURONAL SYSTEMS 17Fig. 3. – Network with a bow-tie structure, resembling the World Wide Web topology.Fig. 4. – Network structure resembling a metabolic pathway.3. – ConclusionsNetwork generation and evolution is still an unsolved problem, and there is not evi-dence that a final word would be ever possible about. The model here proposed showshow a dynamic approach to the problem is possible and, from few reasonable hypotheses(the concept of local competition and collaboration) it is possible to describe a lot ofdifferent phenomenological structures (network topologies). The concept of competitionis widespread in a lot of different models, from biology to economy, due to the intrinsicminimization of a cost functional, and such behavior is surely common to any systemwith limited resources.∗ ∗ ∗D.R. was supported by “Progetto Strategico d’Ateneo 2006”; D.R. and G.C. byINFN SHEILA and EXCALIBUR National Grants, I.Z. from MIUR FIRB ITAL-BIONET (RBPR05ZK2Z-001), L.M. from MIUR LITBIO (RBLA0332RH) and EGEE-III, NET2DRUG European projects.18 G. CASTELLANI, E. VERONDINI, E. GIAMPIERI, ETC.REFERENCES[1] Erdos P. and Renyi A., Public. Math., 6 (1959) 290.[2] Albert R. and Barabasi A. L., Rev. Mod. Phys., 74 (2002) 47.[3] Watts D. J. and Strogatz S. H., Nature, 393 (1998) 440.[4] Bienenstock E. L., Cooper L. N. and Munro P. W., J. Neurosci., 1 (1982) 32.[5] Intrator N. and Cooper L. N., Neural Networks, 5 (1992) 3.[6] Bishop C. M., Neural Networks for Pattern Recognition (Oxford University Press) 1995.[7] Remondini D., Tieri P., Valensin S., Verondini E., Franceschi C., Bersani F. andCastellani G. C., Lect. Notes Comput. Sci., (2006) 286.[8] Broder A., Kumar R., Maghoul F., Raghavan P., Rajagopalan S., Stata R.,Tomkins A. and Wiener J., Comput. Networks, 33 (2000) 309.

Large-scale modelling of neuronal systems

http://eprints.bice.rm.cnr.it/16476/1/ncc9372.pdf

Large-scale modelling of neuronal systems

Abstract

Similar works

Full text

Available Versions

Scientific Open-access Literature Archive and Repository