Making use of biologically plausible artificial neural networks that implement Grossberg’s presynaptic learning rule, we simulate the possible effects of calcium dysregulation in the neuron’s activation function, to represent the most accepted model of Alzheimer's Disease: the calcium dysregulation hypothesis. According to Cudmore and Turrigiano calcium dysregulation alters the shifting dynamic of the neuron’s activation function (intrinsic plasticity). We propose that this alteration might affect the stability of synaptic weights in which memories are stored. The results of the simulation supported the theoretical hypothesis, implying that the emergence of Alzheimer's disease's symptoms such as memory loss and learning problems might be correlated to intrinsic neuronal plasticity impairment due to calcium dysregulation

Andina de la Fuente, Diego

Lobo Netto, Marcio

Monteiro, Julio L.R.

Ropero Peláez, Javier

Archivo Digital UPM

Using Neural Networks to Simulate the Alzheimer's Disease

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

USING NEURAL NETWORKS TO SIMULATE THE ALZHEIMER'S DISEASE 
JÚLIO L. R. MONTEIRO, University of São Paulo, Brazil, julio.monteiro@poli.usp.br
MARCIO LOBO NETTO, University of São Paulo, Brazil, lobonett@lsi.usp.br
DIEGO ANDINA, Technical University of Madrid, Spain, andina@gc.ssr.upm.es
JAVIER ROPERO PELÁEZ, University of São Paulo, Brazil, fjavier@usp.br
ABSTRACT
Making  use  of  biologically  plausible  artificial  neural  networks  that  implement  Grossberg’s 
presynaptic  learning  rule,  we  simulate  the  possible  effects  of  calcium  dysregulation  in  the 
neuron’s activation function, to represent the most accepted model of Alzheimer's Disease: the 
calcium dysregulation hypothesis. According to Cudmore and Turrigiano calcium dysregulation 
alters the shifting dynamic of the neuron’s activation function (intrinsic plasticity). We propose 
that this alteration might affect the stability of synaptic weights in which memories are stored. 
The results of the simulation supported the theoretical hypothesis, implying that the emergence of 
Alzheimer's disease's symptoms such as memory loss and learning problems might be correlated 
to intrinsic neuronal plasticity impairment due to calcium dysregulation.
KEYWORDS:  Artificial  Neural  Networks,  Computer  Simulation,  Alzheimer's  Disease, 
Calcium Hypothesis, Neural Plasticity, MatLAB.
1. INTRODUCTION
Although the Alzheimer's  Disease (AD) was first  documented by Alois Alzheimer[3] one 
hundred  years  ago,  the  understanding  of  its  ultimate  cause  still  represents  a  mystery  to 
neurobiology.  There  are  many theories  for  the  disease  mechanism,  including  the  cholinergic 
hypothesis  [4],  the  Tau  hypothesis  [5],  the  Amyloid  hypothesis  [6]  and  the  recent  Calcium 
dysregulation hypothesis [7]. The calcium dysregulation hypothesis (CDH) of brain aging and 
AD [1],[8],[9],[10], suggests that aging alters brain Ca2+ regulation, impairing neuronal function 
and leading to neurodegeneration. In this article, we present a formal method of simulating the 
effects  of  the  dysregulation  of  calcium  in  an  artificial  neural  network  to  allow  a  better 
understanding on how the AD takes place.  We do not  intend,  however,  to give an extensive 
account of the processes in which calcium is involved, as this can be found elsewhere [1][2][6][7]
[8][9]. We merely will focus our study in the effect produced by calcium dysregulation on the 
dynamic of the neuron’s activation function and in the consequences of this effect on memory 
and learning.
In the following section (section 2) we present an introduction to the regulatory mechanisms of 
neuronal plasticity,  to allow the understanding of the biological process involved in the CDH 
(section 3). In section 4 we introduce the neural network computational model that makes use of 
the neuronal plasticity properties explained in section 3. The experimental setup of the simulation 
utilizing MathWorks's MatLAB is also explained. Section 5 presents the results obtained when 
intrinsic  neuronal  plasticity  is  hampered,  according  to  the  calcium dysregulation  hypothesis. 
Finally section 6 compares experimental and expected results, concluding that the results of the 
simulation are consistent with the memory loss and learning impairments in AD.
2. HOMEOSTATIC MECHANISMS OF NEURONAL PLASTICITY
Neurons display complex adaptable responses for regulating the input stimuli from synapses 
and also the output generated in the cellular soma.
The regulation of an internal environment so as to maintain a stable, constant condition is also 
called homeostasis, which was a term coined by Walter Bradford Cannon in 1932. 
In this  paper two relevant regulatory or homeostatic mechanisms  of neuronal activity will  be 
presented:  metaplasticity  and  intrinsic  plasticity.  For  a  correct  understanding  of  these  two 
mechanisms we will start first with an introduction to synaptic plasticity.
2.1 Synaptic plasticity
The  transmission  of  information  between  neurons  is  mediated  by  molecules  known  as 
neurotransmitters that act on synapses, and interact with ionic channels residing in the synaptic 
membrane,  thereby  allowing  the  inflow  or  outflow  of  positive  and  negative  ions  such  as 
potassium,  sodium,  chlorine  and calcium.  Synaptic plasticity,  refers  to  the  modulation of  the 
efficacy of  information  transmission  between  neurons,  being  related  to  the  regulation  of  the 
number of ionic channels in synapses.
The first model of synaptic plasticity was postulated by Hebb and is known as the Hebb rule[11] 
that is stated as follows: when two neurons fire together wire together or, in other words, the 
synaptic strength between neurons with a correlated firing tends to increase. Mathematically the 
change in the synaptic strength (synaptic weight) between neurons  i and  j is calculated by the 
product of the output of neuron i and the input Ij (which corresponds to the output of neuron j) 
multiplied by a learning constant.
  (1)
Some  authors  proposed revised versions  of  the  Hebb's  rule,  taking into account  more  recent 
biological studies [12]. The formulation that was adopted for our simulation of synaptic plasticity, 
due to its biological plausibility,  is the Grossberg's presynaptic learning rule [13] either in its 
incremental or in its probabilistic version.
The incremental version of the pre-synaptic rule is as follows:
   (2)
According  to  Minai  [14],  this  incremental  version  of  the  pre-synaptic  rule  is  asymptotically 
equivalent to the following probabilistic version (Eq.3), where the synaptic weight between two 
neurons is the conditional probability of the output neuron’s firing, given that the input neuron 
have previously fired.
   (3)
2.2 Synaptic metaplasticity
One of  the  important  biological  characteristics  of  the  presynaptic  rule  is  that  it  also models 
metaplasticity,  which is  an important homeostatic mechanism of neurons.   It  slows down the 
process of weight increment or decrement, making more difficult for the neuron to become either 
inactive or saturated.
The property of metaplasticity [15] is shown in Fig.1a representing a family of curves in which 
each  curve  yields  the  variation  of  weight  given  the  neuron’s  activation.  The  parameter  that 
defines what curve must be used is the value of the synaptic weight.  According to this figure, for 
higher values of the weight the curves are more elongated to the right.
This means that in synapses with higher weights, the interval in which the variation of weight is 
negative is broader, thereby favoring synaptic depression. The opposite takes place in the lower 
weight curves.
jiij IOw ε=∆
( )jiij IOPw /=
)( ijijij wOIw −=∆ ε
Synaptic metaplasticity is a homeostatic mechanism because it regulates weight variation, down-
regulating  weight  increment  in  synapses  with  initial  high  weights  and  up-regulating  weight 
increment in synapses with initial low weights.
2.3 Intrinsic plasticity
Although synaptic metaplasticity makes it difficult for synaptic weights to become either null or 
saturated, it does not totally preclude either of these two extreme situations. For totally precluding 
the possibility of either weight annihilation or saturation, another important homeostatic property 
of real neurons should be taken into account: the so-called intrinsic plasticity. 
Intrinsic  plasticity,  demonstrated  in  Fig.  1b,  regulates  the  position  (rightward  shift)  of  the 
neuron’s  activation function,  usually modeled as a sigmoidal  function,  according to previous 
levels of activity:
(4)
In which O is the output probability of the neuron and “a” is the activation function given by the 
sum of synaptic contributions.
Intrinsic plasticity (Fig.  1b) was modeled according to the following equation that  yields  the 
position of the sigmoid in terms of the previous position shift-1 and the previous sigmoid output.
 (5)
Parameter ξ  determines the velocity of the sigmoid shifting. This equation means that the more 
the neuron keeps firing, the higher will be the rightward shift of the activation function, leading to 
a moderation of the neurons’ firing probability in the future. Conversely, if the firing probability 
is low, the sigmoid will move leftwards, thereby increasing the probability of the neuron’s firing 
in the future.
We simulate the effects of the calcium dysregulation by setting ξ  in Eq. (4) to zero, so that the 
shifting dynamics of the sigmoid is stopped, destroying the effect of intrinsic plasticity. In this 
ξ
ξ
+
+
=
−−
1
11 tt
t
shiftO
shift
( )shiftae
O
−−+
= 251
1
              (a) (b)
Fig. 1: The curves in (a) represents synaptic metaplasticity, in which more elongated curves  
are obtained for higher initial synaptic weights; in (b), intrinsic plasticity consists in the shift  
of the activation function  according to higher or lower levels of neural firing 
way, without the compensatory mechanism of intrinsic plasticity, weights are free to increase or 
decrease without bound.
3. THE CALCIUM DYSREGULATION HYPOTHESIS
The calcium dysregulation hypothesis  (CDH) is  referred to a  broad category of calcium-
dependent  neural  processes  related  to  the  gradual  impairment  of  cognitive  abilities  in  AD, 
especially those related to memorizing and learning. Thibault and colleagues [1] enumerate some 
of these calcium-dependent processes like the increase in slow after-hyperpolarization (AHP), 
reduction of long-term potentiation (LTP), enhancement of  long-term depotentiation (LTD) and 
impairment of short-term frequency facilitation (FF). In most of these cases, the role of Ca2+ is 
correlated  to  changes  in  plasticity,  although the  exact  mechanism explaining  this  correlation 
remains  uncertain.  However,  in the  case  of  intrinsic plasticity,  Cudmore  and Turrigiano [19] 
showed that there is a clear relationship between intrinsic plasticity and Ca2+ influx. Reducing 
extracellular calcium, prevents the leftward shifting of the activation function curve. In this way, 
if calcium influx is altered, a dysregulation in the shifting dynamic of the activation function 
takes  place.  In the following simulations we will  show that  the  dysregulation of the shifting 
dynamic of the activation function leads to the alteration of synaptic weights, where memories are 
encoded.  Synaptic weight destabilization might be an explanation of the impairment in memory 
retrieval and in learning in AD.
The same equations used in this section were described in [15] and used successfully to describe 
and simulate other type of neural disorders[16][17][18].
4. COMPUTER SIMULATION
For allowing a better understanding of the relationship between calcium dysregulation and 
cognitive loss, a computational simulation was performed.
In this simulation, a small artificial neural network of 30 sparsely connected neurons was studied. 
The synaptic weights of each one of these neurons were modified according to the presynaptic 
learning rule Eq. (3). The shift of the sigmoidal activation functions of these neurons is calculated 
according to Eq. (5). The neuronal network is exposed repetitively to a series of 5 patterns that 
were presented to 5 neurons considered as “input” neurons. The whole presentation of these 5 
patterns was considered an “epoch” of patterns, according to the neural networks terminology.
After  150  epochs,  the  network  weights  reached  either  static  or  dynamic  equilibrium  in  a 
predictable  behavior.  For  simulating  the  starting  point  of  the  Alzheimer  Disease,  the  shift 
variations of all neurons in the network are suddenly hampered by setting the shifting velocity ξ  
to zero. In this way, we simulate the cessation of intrinsic plasticity dynamics caused by calcium 
dysregulation.
5. RESULTS
The simulation was written in  MathWorks's  MatLAB.  The network is  implemented  as  a 
30x30 sparse matrix of weights, with the probability of having positive weights equal to 0.10 and 
a  probability  of  having  negative  weights  (representing  inhibitory  neurons)  equal  to  0.30. 
According  to  biology,  neurons  in  the  model  are  either  excitatory or  inhibitory,  without  any 
neuron being excitatory and inhibitory at the same time.
All neurons have the same sigmoid function given by Eq. (4) for calculating the action potential 
probability from incoming synaptic  activation.  The shift  of  the sigmoid is  initially set  to 0.5 
whereas ξ  is set to 0.01 during the first 150 epochs of the simulation.
At the beginning of each simulation, five input patterns are randomly generated and continuously 
input  to 5 neurons acting as inputs for the network during 150 epochs.  After  this number  of 
iterations, the shifting velocity ξ  is zeroed, stalling the sigmoid dynamic.
The results are a set of graphs showing the outputs of each of the 30 neurons in the network as the 
simulation progresses. 
The simulation yielded three different categories of results that are exemplified by the following 
pair of graphs.
Each graph represents the variation of the outputs and weights in the network along the different 
epochs, indicating the outputs of each of the 30 neurons during the presentation of the last pattern 
of each epoch.
In 80% of the random simulated networks, synaptic weight lose their stability after the sigmoid 
stall, resulting in graphs similar to Fig. 2a.
Around 25% of the random simulated networks exhibits an oscillatory, burst-like behavior during 
the first phase of the simulation. This behavior is lost after the sigmoid stall in the second phase 
of the simulation, as is exemplified in Fig. 2b. No network exhibiting burst behaviour continues 
bursting after the stall of the sigmoid dynamics. 
However, 20% of the randomly initialized networks continue behaving as in the initial phase, so 
that even after the sigmoid stall, no significant change in the network is perceived, as exemplified 
in Fig. 2c. This is consistent with the fact that a reduce number of patients with Azheimer disease 
do not exhibit memory or learning impairments.
6. CONCLUSIONS
In this  paper we have shown that  calcium dysregulation might  be the starting point  of  a 
cascade of events leading to the lose of synaptic weight stability and, thereby, to the lose of the 
memories stored in synaptic weights. This might explain memory deficits in Alzheimer’s disease 
patients.
According to Cudmore and Turrigiano [19], the inflow of calcium is determinant  in intrinsic 
plasticity. Intrinsic plasticity is a homeostatic mechanism that shifts the activation function to the 
right or to the left depending on the degree of synaptic activation of the neuron, so that the neuron 
is never zeroed or saturated. At the same time intrinsic plasticity leads to a stability of weights. 
When the compensatory mechanism of intrinsic plasticity is disrupted synaptic weights may lose 
their stable values.
We  have  tested  this  hypothesis  by training  a  neural  network  until  weights  reached  a  stable 
situation.  Afterwards,  the  shifting  dynamics  of  the  activation  function  is  hampered.  The 
computational model shows that, although in some cases synaptic weights remain stable, in most 
of the cases they enter in a situation of instability. These results are confirmed with networks of 
different  sizes  (networks  of  30,  50  and  100  neurons  were  tested  with  similar  results)  with 
Fig. 2: In (a), the sigmoid stall disrupts the synaptic weights severely; in (b), the burst  
activation behavior is lost after sigmoid stall; (c) shows a particular random network where 
the effect of the sigmoid stall is almost irrelevant
different architecture of connections (in each simulation synaptic weights were initialized by a 
randomly generated sparse matrix in which we define the percentage of excitatory or inhibitory 
connections)  and  with  different  input  patterns  (that  were  also  randomly  initialized  at  the 
beginning of each simulation).
These results show that calcium dysregulation that is correlated to the impairment of intrinsic 
plasticity, leads to synaptic instability, which is consistent with cognitive deficits in AD.
7. REFERENCES
[1]  Thibault  O,  Hadley R,  Landfield PW, Elevated Postsynaptic  [Ca2+]i and L-Type  Calcium 
Channel  Activity  in  Aged  Hippocampal  Neurons:  Relationship  to  Impaired  Synaptic 
Plasticity, The Journal of Neuroscience, Vol.21, No.24, 2001, pp. 9744-9756.
[2]  Desai  NS,  Homeostatic  plasticity  in  the  CNS:  synaptic  and  intrinsic  forms,  Journal  of  
Physiology, Vol.97, 2003, pp.391-402.
[3] Alzheimer  A, Über eine eigenartige Erkrankung der Hirnrinde,  Allgemeine Zeitschrift  fur  
Psychiatrie Vol.64, 1907, pp.146-148.
[4] Shen ZX, Brain cholinesterases: II. The molecular and cellular basis of Alzheimer's disease, 
Med. Hypotheses 63 (2), 2004, pp. 308-21.
[5] Schmitz C, Rutten BP, Pielen A, et al, Hippocampal neuron loss exceeds amyloid plaque load 
in a transgenic mouse model of Alzheimer's disease,  American Journal of Pathology, Vol.
164, No.4, 2004, pp.1495-1502.
[6] Iijima K, Liu HP, Chiang AS, et al, Dissecting the pathological effects of human Abeta40 and 
Abeta42 in Drosophila: a potential model for Alzheimer's disease, Proc. National Academy 
of Sciences U.S.A. 101 (17), 2004, pp.6623-6628.
[7]  O’Day,  Danton  H.  and  Michael  A.  Myre,  Calmodulin  Binding  Domains  in  Alzheimer’s 
Disease  Proteins:  Extending the  Calcium Hypothesis,  Biochemical  Biophysical  Research 
Communications 320, 2004, pp.1051-1054.
[8] Khachaturian ZS, Towards theories of brain aging, Handbook of studies on psychiatry and old 
age (Kay D, Burrows GD, eds), Amsterdam: Elsevier, 1984, pp 7–30.
[9] Landfield PW, Pitler TA,  Prolonged Ca 2+ dependent afterhyperpolarizations in hippocampal 
neurons of aged rats. Science 226, 1984, pp. 1089–1092.
[10] Michaelis ML, Johe K, Kitos TE, Age-dependent alterations in synaptic membrane systems 
for Ca2+ regulation. Mech Aging Dev 25, 1984, pp. 215–225.
[11] Hebb DO, The organization of behavior. New York: Wiley, 1949.
[12]  Brown TH,  Kairiss  EW,  Keenan  CL,  Hebbian  Synapses:  Biophysical  Mechanisms  and 
Algorithms, Annual Review in Neurosciences, Vol. 13, 1990, pp. 475-511.
[13] Grossberg S, Adaptive pattern classification and universal recoding: I. Parallel development 
and coding of neural feature detectors. Biological Cybernetics, Vol. 23, 1976, pp.121-134.
[14] Minai AA, Levy WB, The Dynamics of Sparse Random Networks, Biological Cybernetics, 
Vol. 70(2), 1993, pp.177-187.
[15] Pelaez JR,  Simoes MG, A Computational Model of Synaptic Metaplasticity.  Proceedings 
of the International Joint Conference of Neural Networks 1999, Vol.1, 1999, pp. 6-11. 
[16] Nascimento AN, Andina D, Peláez FJR, Towards a Neural-Networks Based Therapy for 
Limbs Spasticity,  Bio-inspired Modeling of  Cognitive Tasks,  Lecture Notes in Computer 
Science, Vol. 4527/2007, 2007, pp. 124-131.
[17] Peláez FJR, Towards a neural network based therapy for hallucinatory disorders.  Neural  
Networks, 2000 Special Issue 13(2000), 2000, pp. 1047-1061.
[18] Granado LC, Ranvaud R, Peláez FJR, A Spiderless Arachnophobia Therapy: Comparison 
between Placebo and Treatment Groups and Six-Month Follow-Up Study, Neural Plasticity, 
Vol. 2007, Article ID 10241, 2007, 11p, doi:10.1155/2007/10241.
[19] Cudmore, R.H. & Turrigiano, G.G. Long-Term Potentiation of Intrinsic Excitability in LV 
Visual Cortical Neurons. Journal of Neurophysiology, 92, 2004, pp. 341-348.


http://oa.upm.es/3695/1/INVE_MEM_2008_56545.pdf

Using Neural Networks to Simulate the Alzheimer's Disease

Abstract

Similar works

Full text

Available Versions

Archivo Digital UPM

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid