We obtain sufficient conditions for the global asymptotic stability of the equilibrium of a class of neural networks with inter-neuronal transmission delays

Gopalsamy, Kondalsamy

Flinders Academic Commons

Convergence in neural networks with interneuronal transmission delays K.  Gopalsamy A b s t ~ a c t -  We obtain sufficient conditions for the global asymptotic stability of the equi- librium of a class of neural networks with in- terneuronal transmission delays. I .  INTRODUCTION A neural network is a network of interconnected elementary units which have limited characteristic properties of real (or biological) neurons. Each unit is capable of receiving many inputs, some of which can activate the unit while other inputs can inhibit the activities of the unit. The neuron-like elemen- tary unit computes a weighted sum of the input,s it receives and fires (or produces) a single response and sends the response down along its ‘axon’, when the weighted sum exceeds a certain threshold. In- terst in the dynamical characteristics of models of neural networks has been steadily increasing during the last 40 years. In this article we consider the following model of a neural network with n units: t > 0 ,  i = 1 , 2 , 3  , . . . ,  n (1) in which xi(t) corresponds to the membrane poten- tial of the i-th unit a t  time t ;  fj(zj) denotes the conversion of the membrane potential of the j-th unit into its firing rate; wij denotes the strength of the j- th unit on the i-th unit; rj corresponds to the transmission delay along the axon of the j-th unit; the constant pi  denotes the external bias or clamped input from outside the network to the i-th unit; bi represents the rate with which the i-th unit will reset its potential t o  the resting state in isola- tion when disconnected from the network and ex- ternal inputs. There exists an extensive literature on various as- pects of systems of the form (1) with and without time delays. For details of literature related to mod- els of the form (l) ,  we refer to [ I ,  5, 6 ,  12, 10, 7 ,  11, The  author is with the School of Information Science and Tech- nology, Flinders University, G . P  0 Box 2 1 0 0 ,  Adelaide SA 5001,  Australia 2, 81 and the references cited therein. For a brief dis- cussion of the significance of time delays in ( l ) ,  we refer the reader to the above cited articles of [5, 61. The purpose of this article is to derive a set of suf- ficient conditions on the parameters b ; ,  w,j, (i, j = 1 , 2 , .  . ., n) ,  i . e .  the network architecture such that a temporally constant external input vector p = ( p l , p 2 , .  . . ,pn)  is encoded as an equilibrium of the system (1) and the network can associatively recall such coded signals by dynamically converging to the equilibrium and such a convergence is independent of the initial values of the phase or system variables. Results of this type have been obtained for systems without delays in the references cited above. Neural networks which converge to equilibria independent of initial values for given external inputs are said to  be stable encoders of the inputs. There is some ev- idence in real biological neural processes for stable encoding of external stimuli (see [13]). 11. CONVERGENCE T O  EQUILIBRIA Equations of the form (1) are known as delay differ- ential equations and are usually supplemented with initial values of the type zi(s) = d i ( s ) ,  4i : [-.I 01 ++ (-m, m), i =  1 , 2 ,  . . . ,  n where 4; is a continuous function; together with such initial values, one can show by the method of steps, that solutions to  (1) exist a t  least locally on an interval of the form [0, t o )  and then by the method of continuation, one extends the interval of existence to [0, m) provided the solutions remain finite. For an extensive range of applications of systems like ( l ) ,  we refer to the recent monograph of the author In the following we assume that each of the output responses f j ,  ( j  = 1 , 2 , .  . . ,  n) is a smooth, strictly monotone increasing function with the following prop- erties: H1 f j  : R H R, f j  is continuously differentiable; H2 f, is bounded on R; H3 0 < - 5 1 ,  j = 1 , 2 ; . . , n ;  U E (-m,oo). [41. 0-7803-1901-X/94 $4.00 01994 IEEE 976 We shall also assume that the intraneuronal cellular or response delays are negligible so that the system (1) is modified to the form -- dzi ( t )  - -bizi(t) + wi i f i ( z i ( t ) )  dt n *=1 j # a  i =  1 , 2 , . . . , n  . (2) The system (2) is autonomous and we are interested in the existence of an equilibrium for (2). Such an equilibrium denotes the coded signal corresponding to the input vector p = (p1 , p z ,  . . . , pn). The unique- ness of the coded signal will follow from the global asymptotic stability to be established below. If z* = (z;, z;, . . . , z i )  denotes an equilibrium of (2), then z* satisfies the nonlinear algebraic system of equations The system (4) can be written in vector-matrix no- tation in the form T* = F ( z * )  = B f ( z * )  + q.  (5) Thus e* is a fixed point of the map F : R" I-+ R". The existence of a fixed point of the map F can be shown by the well known Brouwer's fixed point theorem. For instance let 52 be a hypercube defined by i2 = { 32 E Rn 1112 - qllm I IlBll M }  I ( 6 )  M = suPlfi(s)l. 8 It is found from (5) and ( 6 )  that l lF(z> - Qllm = llBf(z)llm I IIBllcollf(~~lloo I IIBlImM. (7) It follows from (5)-(7) that the map F is continuous and maps a bounded closed and convex set 52 into itself; hence by the Brouwer's theorem, the map F has at  least one fixed point say z*. We note that Brouwer's theorem does not guarantee the unique- ness of the fixed point. However, we obtain suffi- cient conditions on the connection matrix W = [wij] which will guarantee not only the uniqueness of the coded signal but also its recall by its global asymp- totic stability. We first observe that all solutions of (2) satisfy differential inequalities of the form 5 -biai(t)  +Pi  (8) d z i ( t )  - b , z ; ( t )  -pi 5 -dt n i =  1 , 2 ,  . . . ,  n It is a consequence of (8) and (9) that solutions of (2) remain bounded for all t 2 0. We need the following preparation for the proof of our main result below. We have from (2) that - [ a $ )  d - e;] dt We let , i =  1 , 2  , . . . ,  n. (11) x i @ )  = + y i ( t )  g i (y i ( t ) )  = f i ( y i  + e;) - f i ( x r )  and note that (10) can be written as follows: n We find that to prove the global asymptotic stability of z*, it is sufficient to prove the global asymptotic stability of the trivial solution of (12). Theorem 1 Suppose that the response functions f,, i = 1 , 2 , .  . . , n  satisfy the hypotheses H l - H 3  above. Assume furthermore that the system parameters b i ,  wij ,  i , j  = 1 , 2 , .  . . , n satisfy the following: J f :  Then the trivial solution of (12) is globally asymp- totically stable. Proof. From the boundedness of solutions of (2), it followp that solutions of (12) remain bounded on 977 (0, CO). We shall prove that under the conditions (13) where Consider a Lyapunov functional V(y)(t) = V(yi, ~ 2 ,  . . . , Yn)(t) defined by J # I  From the properties of fi and hence of g;, i t  follows that v ( Y i , ~ 2 , . . - , Y n )  > 0 for ( Y ~ , Y Z , . . . , Y ~ )  # ( O , O ,  . . . ,0)  and V(y) is bounded for t E [0, m). Cal- culating the rate of change of V along the solutions of (W,  n -3 A consequence of (18) is that I V(Y1 1 ~ 2 ,  . . . , Yn)(O). (20) It follows from the boundedness of V(y1 , y2, . . . , yn)(t) for t E [O,m) and (20) that  g;(yi(s)) ds < CO, i = 1 , 2 , .  . ., n. (21) The boundedness of xi, f i ,  y, i = 1 , 2 , .  . . , n im- ply t h a t o f g i , ~ ;  and % , i =  l , 2 ,  . . . ,  n lead ing to  the uniform continuity of g:(y,(t)), i = 1 , 2 , .  . ., n on [0, CO). By Barbalatt’s lemma (see [4]) it follows that lim y;(t) = 0, i = 1 , 2 , .  . . , n .  (24) If (24) does not hold, letpi = limsup,,, yi(t) and suppose that pi # 0; then there exists a sequence say {tr’} -+ CO as k + 00 and We estimate the right side of (16) using the ele- ing which are consequences of our hypotheses on f,, i =  1,2 ,  . . . ,  n; mentary inequality 2ab 5 a2 + b2 and the follow- t-02 , i =  l 1 2 , . . . , n .  pi = lim y i ( t r ) ) ,  i =  1 , 2 , . . . , n ;  (25) We have from (25) by the monotonicity and conti- k - m  (17) I Yi 5 gi(Yi) < 0 for ~i < 0 gi(0) = 0 0 < gi(yi) I ~i for Yi > 0 Using (17) in (1611 we estimate the right side of(16) so that nuity o f f ,  that # fi(xcf), i = 192,.  . (26) which contradicts (23). Thus pi = 0, i = 1 , 2 , .  . ., n. Similarly one proves that liminft,, yi(t) = 0, i = 1 , 2 , .  . . , n.  Hence (14) follows and this com- pletes the proof. It is possible to consider continuously distributed delays in (2) by replacing the terms having discrete delays xj(t  - r j )  with terms of the type K j ( ~ ) ~ j ( t  - s ) d s ,  j = 1 , 2 , . . . , n  I” 978 where Kj : [O,oo) ++ [ O , o o )  are continuous and in- tegrable satisfying Jr K j ( s ) d s  = 1 ,j= 1 ,2  , . . . ,  n. (27) Jo sKj (s) d s  < 00 In such a case the system (2) reads + p i ,  i =  1 , 2 , . . . , n .  (28) As before one can show that solutions of (28) re- main bounded; note that the initial conditions for (28) are bounded continuous functions defined on (-00~01. Using (27) it can be found that the equi- librium of (28) coincides with the z’ of (2). The following Corollary provides sufficient conditions for the global asymptotic stability of x* of (28). Corollary 1 Suppose the assumptions of the above Theorem hold; then the equilibrium x* of (28) is globally asymptotically stable in the sense that any solution of (28) satisfies the convergence condition lim z j ( t )  = X I ,  i = 1 , 2 , .  ..,n. (29) t+w Proof. The details of proof are based on a minor modification of those of the above Theorem and are omitted. The sufficient conditions established in the Theo- rem above are explicitly in terms of the parameters of the system and hence are easily verifiable. The conditions mean that each of the intraneuronal sta- bilising feedback acts without delay and the strength of such a stabilising feedback measured by bi - w,, dominates the interneuronal weights of activations and inhibitions measured by Cfl 3 #a  . (Iwjjl + wjjl); in such a case the interneuronal transmission (ax- onal) delays cannot destabilise the network and the network will not embark on delay induced persistent oscillations; the resulting convergence is thus inde- pendent of delays and the global asymptotic sta- bility indicates the convergence is independent of initial values; there is no need to reset the initial values and the system will converge to the relevant equilibria whenever the external inputs are replaced and clamped. REFERENCES [l] A. F. Atiya, Learning on a general network. In D.Z. Anderson (Ed.) Neural information processing systems (pp. 22 - 30). New York; American Institute of Physics, 1988. [2] T. A. Burton, Neural networks with memory, Applied Math. Stoch. Anal., vol. 4, pp. 313-332, 1991. [3] M. A. Cohen and S. Grossberg, Absolute sta- bility of global pattern formation and paral- lel memory storage by competitive neural net- works. IEEE Trans. Syst .  Man and Cybemel- ics, SMC vol. 13,pp. 815 - 826, 1983. [4] K. Gopalsamy, Stability and oscillations in delay differential equations of popula- tion dynamics, Kluwer Academic, Dordrecht, Netherlands, 1992. 151 K. Gopalsamy and X. Z .  He, Stability in asym- metric Hopfield nets with transmission delays, IEEE Trans. Neural Networks (in press), 1994. [6] K. Gopalsamy and X. Z .  He, Delay indepen- dent stability in bidirectional associative mem- ory networks , Physica D (in press), 1994. [7] A.  Guez,V. Protopopsecu and J .  Barhen, On the stability, storage capacity, and design of nonlinear continuous neural networks, IEEE Transactions on Syst. Man and Cybernetics, [8] M. W.  Hirsch, Convergent activation dynamics in continuous time networks, Neural Networks, [9] J .  J .  Hopfield, Neurons with graded response have collective computational properties like those of two-state neurons, Proc. National Academy of Sciences, USA, 81,pp. 3088 - 3092, 1984. [lo] D. G. Kelly, Stability in contractive nonlinear neural networks. IEEE Trans. Biomedical En- gineering ,vol. 37, pp. 231 - 242,1990. [ll] C. M. Marcus and R. M. Westervelt, Stabil- ity of analog neural networks with time delay. Phys. Rev. A ,  vol. 39, pp. 347 - 369, 1989. [12] K. Matsuoka, Stability conditions for nonlin- ear continuous neural networks with asymmet- ric connection weights, Neural Networks, vol. 5, [13] 0. Whitsel, M. Favorov, M. Tommerdahl, S. Juliano and D. Kelly, Dynamic processes gov- ern the somatosensory cortical response to nat- ural stimulation, in (Ed.) J .  S.  Lund Organi- zational principles of sensory processing: The Brain’s view of the world, Oxford Uni. press, New York, 1987. SMC vol. 18, pp. 80 - 87, 1988. vol. 2, pp. 331-349, 1989. pp. 495 - 500, 1992. 979 

Convergence in neural networks with interneuronal transmission delays

http://dspace.flinders.edu.au/xmlui/bitstream/handle/2328/25795/Gopalsamy%20Convergence.pdf?sequence=1

Convergence in neural networks with interneuronal transmission delays

Abstract

Similar works

Full text

Available Versions

Flinders Academic Commons