Inspired by biological cognition, CABOT project explores
the ways symbolic processing can emerge in a system of neural cell assemblies (CAs). Here we show how a stochastic meta–control process can regulate learning of associations between the CAs, the neural basis of symbols. An experiment illustrates the learning between CAs representing conditions actions pairs, which leads to CA–based representations of ‘if–then’ rules

Belavkin, R.

Huyck, C.

English

Middlesex University Research Repository

Emergence of Rules in Cell Assemblies of fLIF NeuronsRoman V. Belavkin and Christian R. Huyck 1Abstract. Inspired by biological cognition, CABOT project ex-plores the ways symbolic processing can emerge in a system of neu-ral cell assemblies (CAs). Here we show how a stochastic meta–control process can regulate learning of associations between theCAs, the neural basis of symbols. An experiment illustrates the learn-ing between CAs representing conditions actions pairs, which leadsto CA–based representations of ‘if–then’ rules.1 INTRODUCTIONPreviously, the authors have demonstrated how states in cell assem-bly (CA) neural system can be controlled and used to perform a typ-ical symbolic task (counting) [5]. This work has developed into amuch more ambitious project called CABOT, where the same prin-ciples are applied in a system, based entirely on CAs, that integrateselements of vision, categorisation, natural language processing andlearning in a virtual environment. This paper presents a part of thisproject — learning the connections between different CAs — thatcombines symbolic representations into logical rules.2 OVERVIEW OF THE ARCHITECTUREOur system uses fatiguing, leaky, integrate and fire (fLIF) neu-rons [4], an extension of LIF neurons [6]:Integrate and fire — the neuron ‘fires’ if its action potential,A, ex-ceeds thresholdθ, whereA = (w, x) =Pki=1 wi xi (integrator),w, x ∈ Rk are the weights and the stimuli vectors. The weightswt adapt according to the compensatory learning rule [4], whichis an implementation of Hebbian learning [3].Leak and accumulation of potential,At+1 = Atdt + (wt, xt), wheredt = ∞ if fired at t; d ≥ 1 otherwise.Fatigue makes the threshold dynamic,θt+1 = θt +Ft, whereFt =F+ ≥ 0 if fired (fatigue); Ft = F− < 0 otherwise (recovery).Cell assembliesare reverberating groups of neurons [3], and they arebelieved to be the neural basis of symbols in human mind. Our sys-tem is based on networks of sparsely connected neurons. The topol-ogy of the networks is pre–defined by some random pattern, and itcan be highly recurrent. When enough neurons fire to start the rever-berating circuit, the CA ignites, and itspersistenceis an importantproperty of CAs’ dynamics. The fatigue and recovery rate parame-ters affect the persistence. A CA can be extinguished by another CA,which can ignite due to the change of the external pattern.A network with several CAs encoding a set of external patternsis referred to as amodule. Several modules can be interconnected tocreate more complex systems. For example, a system of 7 modulesand 40 CAs was used to implement a simple counting task [5]. More1 Middlesex University, London NW4 4BT, UKcomplex systems have been used to parse natural language and im-plement finite state automata. The next stage in the development ofthe project is the ability to learn the connections between differentmodules, the focus of this paper.3 STOCHASTIC META–CONTROLAlthough the connections between the correlated cells are strength-ened via Hebbian learning, it is the meta–process that controls whichneurons fire and thus which connections are supported. The meta–process is based on stochastic control of action–selection algorithms,implemented earlier by the authors in cognitive architectures [1] andwhich are based on the following result of information theory.Given utility functionu : Ω → R, the goal is to find probabilitydistribution,p on Ω, that maximises the expected utilityEp{u} =(p, u) =Ppiui under additional constraints. This distribution isp(ω) = q(ω) eβu(ω)−Γ(β) (1)where q(ω) is the reference (prior) distribution,Γ(β) =lnPΩ q(ω)eβu(ω), andβ is the Lagrange multiplier, defined fromconstraints on information (I p, q) ≤ I < ∞) or on the expectedutility (Ep{u} ≥ U > −∞):β(U) =dI(U)dU, I(U) = supβ[Uβ − Γ(β)] (2)Functionβ(U) is strictly increasing, and forβ > 0 the optimal dis-tribution (1) has non–zero values (p(ω) > 0) for all ω ∈ Ω such thatu(ω) > −∞. Thus, the optimal distribution describes stochastic pro-cess, where allω are randomised by the control parameterβ > 0, orits inverseT = β−1 called thetemperature.Value–Explore Topology.Problems of optimal control often involvemaximisation of utility over setΩ = X × Y , whereX is the set ofobservations (e.g. goals), andY is the set of controls (e.g. actions). Inour system, these sets are represented by two modules, Goals and Ac-tions, where CAs represent conditions and actions respectively. Thus,ω ∈ Ω are condition–action pairs(x, y) ∈ X × Y .Value // Explore Goal1...Goalm//////Act 1...Act nInitially, the modules are set up with excitatory connections from ev-ery x ∈ X to all y ∈ Y . Thus, given some goal, any action can betriggered. Due to the Hebbian learning, the connectionsx → y be-tween CAs that have fired together are reinforced, giving the pair ahigher chance to ignite in the future. Thus, due to Hebbian learning,the system can learn some random relationR ⊂ X×Y (set of rules),which may not be optimal. Learning of only a particular (optimal) re-lation is supported by the meta–process that involves two additionalmodules: Value and Explore.The activity of the Value module represents the values of utility(higher activity corresponds to higher utility). The average activityof the module corresponds to constraintU in equation (2). The inputof the module can be configured according to the application. Forexample, it may receive inputs from the sensory system representingagent’s preference on the states of the environment.The purpose of the Explore module is to randomise the activityof the Action module. Cells in this module are spontaneously firing,and the module sends excitatory connections to all CAs in the Actionnet. Thus, the Explore module can trigger randomly any Action CA,and this process has no memory. The module implements the effectof parameterβ > 0 in equation (1) (or the temperatureT = β−1).The Value module sends inhibitory connections to Explore, so thathigh activity of Value inhibits the activity of Explore. This imple-ments the monotonic relation between constraintU andβ in equa-tion (2), and it allows for a very simple yet effective learning scheme.If a particular goal–action pair(x, y) results in a high utility, then theValue module inhibits Explore, and the(x, y) pair is allowed to per-sist longer. Since high utility pairs(x, y) on average co–fire longerthan low utility pairs, their connections increase relative to others dueto the compensatory Hebbian learning rule.This way, the meta–process supports learning of the optimal re-lation R ⊂ X × Y . As a result, the average activity of the Valuemodule (U ) increases with time, while the activity of the Exploremodule (T = β−1) decreases. The system makes a transition fromstochastic to an almost deterministic rule–based system.The biological plausibility of this topology is supported by studiesof the reward path and tonically active cholinergic neurons in thebasal ganglia and striatal complex [2]. These neurons account for asmall proportion of the connections, and they are quite uniform andnontopographic. These neurons may play the role of stochastic noise,and their activation is reduced when the reward path is activated.4 EXPERIMENT: LEARNING DICHOTOMIESThe code of the system and the experiment described is available athttp://www.cwa.mdx.ac.uk/CABot/CANT.htmlIn this simple experiment, there are two CAs in the Goal and two CAsin the Action modules. Each module consisted of 800 cells, with 400cells in each CA. The modules were set up with low weight excitatoryconnections from every goal CA to all action CAs, shown by dashedarrows on the left diagram below. The task was to learn two rules,shown by two solid arrows on the right diagram.Goal 1 //___$$IIIIIAct 1Goal 2 //___::uuuuuAct 2Goal 1 // Act 1Goal 2 // Act 2The training procedure consisted of a random presentation of aninput pattern activating one of the goal CAs every 100 cycles. Fig-ure 1 shows the proportion of the correct actions selected (ordinate)as a function of cycles (abscissa). The chart shows the results of fivesimulations. Initially the system makes only half of the choices cor-rectly. After 3000 cycles, the proportion of correct choices increasesto 70–90%. Figure 2 shows the percentage of neurons firing per cycle00.20.40.60.810 50 100 150 200 250 300Cycles (x10)Figure 1. The proportion of correct action choices (ordinate) as a functioof cycles (abscissa). The curves represent results of different trials.00.20.40.60.810 50 100 150 200 250 300Cycles (x10)ValueExploreFigure 2. Activities of the Value and Explore modules in one experiment.in the Value and the Explore modules in one of the experiments. Asdesired, an increase of the Value activity coincides with the decreaseof the Explore.The implementation of the meta–process for rule acquisition in oursystem is an important step in its evolution creating new opportuni-ties and improving our understanding of biological cognition.ACKNOWLEDGEMENTSThis work was supported by EPSRC grant EP/DO59720.REFERENCES[1] R. V. Belavkin, ‘Acting irrationally to improve performance in stochasticworlds’, in Proceedings of AI–2005, the 25th SGAI International Confer-ence on Innovative Techniques and Applications of Artificial Intelligence,eds., M. Bramer, F. Coenen, and T. Allen, pp. 305–316, Cambridge, (De-cember 2005). Springer.[2] R. Granger, ‘Engines of the brain: The computational instruction set ofhuman cognition’,AI Magazine, 27(2), 15–32, (July 2006).[3] D. O. Hebb,The Organization of Behavior, John Wiley & Sons, NewYork, 1949.[4] C. Huyck, ‘Hierarchical cell assemblies’,Connection Science, (2007).[5] C. Huyck and R. V. Belavkin, ‘Counting with neurons, ruleapplicationwith nets of fatiguing leaky integrate and fire neurons’, inProceedingsof the Seventh International Conference on Cognitive Modeling, eds., D.Fum, F. D. Missier, and A. Stocco, Trieste, Italy, (April 2006). EdizioniGoliardiche.[6] W. Maas and C. Bishop,Pulsed Neural Networks, MIT Press, 2001.

Emergence of rules in cell assemblies of fLIF neurons.

https://repository.mdx.ac.uk/download/c615f629a0c66dd35226c412c43bacc2e17237abeeeac5343cf2e3ed26d4f0e4/54186/huyckbelavlin-emergence_of_rules.pdf

Emergence of rules in cell assemblies of fLIF neurons.

Abstract

Similar works

Full text

Available Versions

Middlesex University Research Repository