A programmable analog neural oscillator cell architecture is presented. The proposed neuron circuit is of hysteretic neural nature with its implementation based on operational transconductance amplifiers (OTA's). The hysteresis loop as well as the frequency of oscillation are voltage (or current) dependent. The architecture, which involves two OTA's, a current mirror, a capacitor, a diode, and a resistor is very suitable for monolithic integrated circuits. Experimental results confirm the expected flexibility of the synthetic neuron

Huertas Díaz, José Luis

Linares Barranco, Bernabé

Rodríguez Vázquez, Ángel Benito

Sánchez Sinencio, Edgar

English

idUS. Depósito de Investigación Universidad de Sevilla

756 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 A Programmable Neural Oscillator Cell BERNABE LINARES-BARRANCO, EDGAR SANCHEZ-SINENCIO, SENIOR MEMBER, IEEE, ANGEL RODR~GUEZ-VAZQUEZ, MEMBER, IEEE, AND JOSE L. HUERTAS, MEMBER, IEEE Abstruct -A programmable analog neural oscillator cell architecture is presented. The proposed neuron circuit is of hysteretic neural nature with its implementation based on operational transconductance amplifiers (OTA’s). The hysteresis loop as well as the frequency of oscillation are voltage (or current) dependent. The architecture, which involves two OTA’s, a current +rror, a capacitor, a diode, and a resistor is very suitable for monolithic integrated circuits. Experimental results confirm the expected flexibility of the synthetic neuron. I. INTRODUCTION HE MOST efficient computational system is the hu- T man brain. Biological systems ordinarily perform rea- soning and logcal manipulations beyond the capabilities of our most modern sophisticated computer systems. Many researchers motivated by that efficiency try to mimic the human nervous system to solve a variety of complex engi- neering problems, among them: control systems, knowl- edge processing and signal/image (sensor) processing. A neural system consists of many highly interconnected neu- rons. Since in general neural networks involve a very large number of parallel arrays of basic cells (neurons), they are very appropriate for VLSI technology. Besides, there are some interesting applications where only a limited number of neurons are used, e.g., low-order optimization problems [l], [2], crude respirator control systems simulation [l], [3]. A biological neuron has both excitatory and inhibitory connections. Each neuron has inputs which are the outputs of other neurons as well as external (stimuli) signals, and if the sum of the inputs is greater than a certain threshold level, a firing sequence of pulses is generated, otherwise below t h s  level the neuron will not fire. The number of pulses fired out depends on the intensity and duration of the input pulse, as well as on the inherent characteristics of the neuron. Manuscript received June 24, 1988; revised December 20, 1988. This paper was recommended by Guest Editors R. W. Newcomb and N. El-Leithy. B. Linares-Barranco was with the University of Seville, 41012 Seville, Spain. He is now with the Department of Electrical Engineering, Texas A&M University, College Station, TX 77843. E. Sanchez-Sinencio is with the Department of Electrical Engineering, Texas A & M University, College Station, TX 77843-3128. A. B. Rodriguez-Vasqua and J. L. Huertas are with the Department of Electronics and Electromagnetics, Faculty of Physics, University of Seville, 41012 Seville, Spain. IEEE Log Number 8826710. Synthetic neurons behave in a crude similar way to the actual neurons. In th s  paper, a programmable neuron circuit architecture is presented. The implementation is based on the operational transconductance amplifier (OTA), this voltage (or current) programmable device is very suitable for synthetic neurons. The proposed neuron circuit is of the hysteretic neural-type pulse oscillator [4] with the inhibitory and excitatory properties as function of the input sum exceeding (or not) a firing threshold level. 11. MATHEMATICAL DESCRIPTION OF A HYSTERETIC NEURON A basic first-order state-variable type equation [ 11, [4] simulating (in a primitive way) a neuron is given by dx dt C -  = - G,x - G , H ( x )  - G J J  y = x  (W where U = input, y = output, x = internal state variable; C ,  G,, G,, and G,  are non-negative constants, and H ( x )  describes the hysteresis nature of the neuron model if x > x ,  - H - ,  H ,  I ,  i f x = - x -  o r x = x +  ( 2 )  Observe that there are other neuron models with similar mathematical characterization. For instance, the term - G,H(x)  in other characterizations [2 ] ,  [5] is related to -C,”=,DJ,h(x)  where DJ, is a weighting function, and f,( .) are nonlinear functions imposing constraints and can be considered as threshold-activation functions. In what follows, we rewrite (1) in a form most useful for our circuit implementation, i.e., dx C - = - G G , ( ~ ) ~ + Z , ( x )  dt y = x  (3b) where I , ( x )  is - G , H ( x ) ,  and the input variable U con- 0098-4094/89/0500-0756$01 .OO 01989 IEEE Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. LINARES-BARRANCO er al. : PROGRAMMABLE OSCILLATOR CELL Fig. 1. Equilibrium point characteristics trols the value of GJu), and the - G,u term has been deleted. The equilibrium points are obtained when d x / d t  = 0, this yields Z , ( X )  = G , ( u ) x .  (4) The intersections of the hysteresis function Zo(x)  with resistive (load-line) G,( u ) x  provide the equilibrium points, as shown in Fig. 1. It can be observed from Fig. 1 that point A is a stable point, and point B is an unstable point. The asymmetry of the hysteresis loop is needed to guaran- tee only one stable point. For our circuit implementation we need a mechanism to change the slope G,(u) such that the straight line moves from point A to point A' and becomes unstable when U becomes excitatory. Hence, the circuit oscillates and fires negative pulses when the input signal U exceeds a certain threshold level; t h s  package of pulses is terminated when U falls below threshold. Thus, there is a critical value G, which separates points like A and A', e.g., G,( U )  > G, stable point A ( 5 )  Gx( U )  < G, unstable point A'. Fig. 2 illustrates the time response of this neuron for a current pulse u ( t )  that makes G,(t) to jump between G, and G, so that G, < G, < G,. This input pulse makes the stable equilibrium point A to change to point A' which is unstable. At the initial time of change of U the output of the hysteretic element starts at I+, and, since x cannot change instantaneously, the differential equation charac- terizing the neuron yields dx  I +  C -  = - G,x + If with x ( 0 )  = -. dt Gl (6a) Hence, the time solution initially is given by This equation holds until x ( t )  becomes x ,  = Z+/G,, at which time the output of the hysteretic element changes to - I -  (we assume the ideal case where no time is taken in going from I ,  to - I -  on the hysterisis curve) and let to be this time, with x ( t , )  = I+/G,. Therefore solving for to  I I  I t o '  tl' t2 Fig. 2. Characteristic time response of neuron cell we obtain to=- ln (  C 1-G2/Gl 1 G2 1 - G2/Gc Now the characterizing differential equation becomes (7) which solution is given by This situation is maintained until x(t) becomes - x-, and I o ( x )  then changes back to + I+ to return to point A and, hence to complete the cycle that will generate a neural-type pulse output. Thus x ( t , )  = - x - .  (10) Solving for t ,  yields C t ,  = - In G, I'G, 1+- I- G, G2 . 1 - - x -  I- Now the differential equation becomes (6a) again dx dt C- = - G,x + I+ but with x(0) = - x -  (12a) and the solution is given by The time at which Io(x) changes from I +  to - I -  occurs Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. 758 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 - Fig. 3. Block diagram of neural-pulse simulator. when in (12b) I +  x ( t , )  = - GC thus t ,  results If T is the duration of the input pulse U and np is the number of pulses generated by the neuron, the next in- equalities hold: to + ( n p  - l ) ( t , +  t , )  < T <  t o +  n,( t ,  + t , ) .  (15) Hence, if one were to hold G, constant at G,<Gc then the number of pulses that would occur while G, remains constant is n,, = Int (:;;:)+1 -where Int(.) produces the integer part of the function ( ( T -  t o ) / ( t , +  t2)). 111. A PROPOSED NEURAL PULSE SIMULATOR A block diagram of the proposed neural-type circuit which implements (3a) and (3b) is shown in Fig. 3. In our circuit, the outputs of the hysteretic element I o ( x )  and of the externally controllable linear amplifier G,(u) are cur- rents, while their inputs are voltages. Accordingly, the integrator is built with a simple capacitor and the current summer element simply implements KCL and, hence needs no additional circuitry. In order to implement the transconductance hysteretic element to yield Zo(x),  we used a differential input transconductance comparator with multiple outputs all taking values as shown in Fig. 4. An antimetric transconductance-hyster- etic element can be implemented as shown in Fig. 5 where R =l /Gc,  and G, is as defined before. Furthermore, i I -  Fig. 4. A transconductance comparator with multiple outputs. (b) Fig. 5. Antimetric transconductance-hysteretic element. (a) Circuit im- plementation. (b) Characteristics of load. f ' h  -E -*x 2 -I - Fig. 6.  Hysteresis characteristics of circuit of Fig. 5. observe that since ui = V,  - x 1) if x < V,  then ui > 0 and I,, = I+  which back biases 2) if x > V,  then ui < 0 and I,, = - I -  and if further we the diode, giving V, = I'R and x < Z'R; choose E, < I - R  then V, = - E, and x > - E,; where I+ = I -  = Ibis [8]. Ibis is the controlling current of the OTA. These observations are summarized in Fig. 6. The other element needed to build up the block diagram of Fig. 3 is an externally current controlled linear transcon- ductance amplifier. A conventional OTA is employed [SI, [9]. The transconductance gain G, is controlled by the bias current U .  This is shown in Fig. 7. Now we can put together the basic elements to form the programmable neural oscillator cell as shown in Fig. 8. The equation describing the behavior of the circuit of Fig. 8 is given by (3) for which x is chosen as the capacitor voltage, ZJx)  has Z + = Z - =  Ibis, and G , = h u  with U the bias current in the bottom OTA in Fig. 8, and h [9] is a proportional constant which depends upon temperature, device geometry, and the process. Neurons are interconnected through synapses. A synapse can be considered as a weighted relation between the output of a neuron and the input received by another one. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. LINARES-BARRANCO et al. : PROGRAMMABLE OSCILLATOR CELL Fig. 9. Hysteresis loop measurement. / I Fig. 7. A linear transconductance amplifier. &A - -  Fig. 8. A hysteretic neural cell implementation. This relation can be inhibitory or excitatory. In the pro- posed neuron circuit, the negative current pulses can be arbitrarily considered to be excitatory signals, while other positive current pulses could be considered the inhibitory signals. This excitatory and inhibitory property of the neuron circuit can be implemented as an integrated circuit version by means of complementary current mirrors for taking output of one neuron to feed other neurons. IV. EXPERIMENTAL RESULTS The circuit of Fig. 8 was built as a test protoboard. The commercial transconductance devices used were the VA703 OTA's. For the linearized OTA implementing G,( U), the linearizing diodes [8] provided within the chip were used as well as an additional resistive attenuator to increase the input voltage swing. The input linear range obtained was about + 3  V for a k5-V power supply. 759 For the transconductance comparator with multiple out- puts, three OTA's (VA703) were used. These three OTA's had their inputs and bias connected together so that they would work like an OTA with three equal outputs. The input impedances of both transconductance devices (the comparator and the linearized one, G,) are not very high, so a voltage buffer had to be added. The values (arbitrarily chosen) of the passive elements were R =1.8 k!d and C = 33 nF. The measured hysteresis loop of Io,' versus x is de- picted in Fig. 9. The bias current of the comparator was adjusted to Ibis = 0.8 mA, and the voltage E, = 0.5 V. The upper trace of Fig. 10(a) shows the output current of the neuron (Io2 = I,,, = I h )  of Fig. 8 when the input U is a square wave (shown in the lower trace) of 4.4 mA. This square wave makes the transconductance of G,( U )  to jump from G ,  = 7.7 X mho, (unstable point) being the width of the pulse (excita- tion time) T = 900 ps. The critical point of G,( U) is given by G, = 1/R = 5.6 x mho. If T, C ,  G,, G,, G,, I+, and E, are known, we can calculate the values of to, t,, t,, (via (7), (11) and (14)) as well as the number of pulses n p  fired during the excitation time T (eq. (16)). mho (stable point) to G, = 1.6 X The calculated values are to = 22 ps t ,  = 75 ps t2 = 89 ps n p  = 6 .  It can be seen from the upper trace of Fig. 10(b), where the voltage across capacitor C is shown for the same input signal U as in Fig. lO(a) that the experimental values of the previous calculated times are approximately: to  = 24 ps t ,  = 80 p~ t ,  = 94 ps np  = 6 .  'Iol was measured across a resistor of 1.0 kQ. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. 760 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS, VOL. 36, NO. 5, MAY 1989 (b) current U .  Time response Vc for a square wave input U .  Fig. 10. (a) Current time response V , / R  for a square wave input Fig. 11. Same as Fig. 10(a) with smaller amplitude and duration of U .  The difference between measured and calculated values is less than 10 percent, which can be due to component tolerances and parasitic elements. Fig. 11 shows the output response V, of circuit of Fig. 8 when the input signal becomes a pulse of smaller mag- nitude and duration, (T-300 ps and n p  = 1  since ( T -  4J/(4 + 1 2 )  <I. v. CONCLUSIONS AND FUTURE WORK An analog synthetic neuron architecture with voltage (or current) programmable properties has been presented. In fact, referring to Fig. 8, the proposed architecture has two degrees of freedom (Ibia and U )  to modify the neuron output; they are functions of the two OTA’s which can be voltage (or current) dependent. This current dependence modifies H ( I + )  and G, in (1). If needed, the weight of the interconnections between neurons (synapses) could be de- termined by means of current mirrors. The addition of current signals from other output neurons or external stimulus at each neuron circuit is accomplished through the controlling terminal of the OTA’s (i.e., see signal U). Contrary to some other analog neural circuits where it is difficult to change their parameters [6 ] ,  this neuron archi- tecture has additional flexibility to modify its parameters according to a certain predetermined algorithm. We are currently working on the MOS integrated version of the proposed neuron architecture taking into account area [7], flexibility, and power consumption. ACKNOWLEDGMENT The authors wish to thank Professor R. W. Newcomb for his valuable comments, suggestions, and encourage- ment in the preparation of this paper. REFERENCES [31 [41 [51 F. C. Hoppensteadt, A n  Introduction to the Mathematics of Neurons. New York: Cambridge Univ. Press, 1286. D. W. Tank and J. J. Hopfield, Simple neural optimization networks: An A/D converter, signal decision circuit, and a linear programming circuit,” IEEE Trans. Circuits Syst., vol. CAS-33, pp. 533-541, May 1986. C. Von Euler, “Central pattern generation during breathing,” Trenh in Neurosci., pp. 275-277, Nov. 1980. G. Kiruthi, R. C. Ajmera, R. Newcomb, T. Yami, and H. Yazdani, “A hysteretic neural type oscillator,” in Proc. IEEE/ISCAS, vol. 3, pp. 1173-1175, May 1983. A. F. Murray and A. V. W. Smith, “Asynchronous VLSI neural networks using pulse-stream arithmetic,” IEEE J .  Solid-State Cir- cuits, vol. 23, pp. 688-697, June 1988. C. Koch, J. Marroquin, and A. Yuille, “Analog ‘neural‘ networks in early vision,” in Proc. Natl. Acad. Sci. (Biophysics), vol. 83, pp. 4263-4267, June 1986. N. El-Leithy and R. W. Newcomb, “Hysteresis in neural-type circuits,” in Proc. IEEE/ISCAS,  vol. 2,  pp. 993-996, June 1988. VTC Incorporated, “High Performance Integrated Circuits,” Data Book (supplement) 1987. R. L. Geiger and E. Sinchez-Sinencio, “Active filter design using operational transconductance amplifiers: A tutorial,” IEEE Cir- cuits Deoices Mag., vol. 1, pp. 20-32, Mar. 1985. B. Linares-Barranco et al., “A novel CMOS analog neural oscillator cell,” to be presented at the ISCAS, May 1989. BernaM Linares-Barranco received the B.Sc. de- gree in electronic physics in 1986, and the M.S. degree in microelectronics in 1987, from the Uni- versity of Seville, Spain. Currently, he is with the Department of Electrical Engineering, Texas A&M University, Texas, where he is working towards the Ph.D. degree. His research interest is in the area of nonlinear analog microelectronic design and neural net- works. Edgar Shchez-Sinencio (S’72-M74-SM83) received the M.S.E.E. de- gree from Stanford University, CA, in 1970, and the Ph.D. degree from the University of Illinois, Champaign-Urbana, in 1973. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. LINARES-BARRANCO et al. : PROGRAMMABLE OSCILLATOR CELL 761 In 1974, he held an industrial Post-Doctoral Since October 1978, he has been with the position with the Central Research Laboratories, Departamento de Electricidad y Electr6nica at Nippon Electric Company Ltd., Kawasaki, the University of Seville, where he is currently Japan. From 1976 to 1983 he was the Head of employed as an Associate Professor. His research Instituto Nacional de Astrofisica, Optica y Elec- interest lies in the fields of analog/digital inte- trbnica, Puebla, Mexico. He was a visiting pro- grated circuit design, neural networks and non- fessor in the Department of Electrical Engineer- linear networks analysis and synthesis. ing at Texas A&M University during the aca- demic years of 1979-1980 and 1983-1984, where he is currrently a Professor. He was the General Chairman of the 1983 16th Midwest S$nposium on Circuits and Systems. He was an Associate Editor of ZEEE Circuits and Systems Magazine (1983-1985), IEEE Circuifs and Devices Magazine (1985-1988) and of the IEEE TRANSACTIONS  CIRCUITS AND SYSTEMS (1985-1987). He is the co-author of the book, Switched-Capacitor Circuits (Van Nostrand- Reinhold, 1984). His current research interest include operational transconductance amplifiers and applications, computer-aided design of solid-state circuits and neural network circuit implementations. rfc J& L. Hwrtas (M’74) received the Licenciado en Fisica and the Doctor en Ciencias Fisicas degrees from the University of Seville, Spain, in 1969 and 1973, respectively. From October 1970 to September 1971, he was with the Philips International Institute, Eind- hoven, The Netherlands, as a Post-Graduate Stu- dent. Since October 1971, he has been with the Departamento de Electricidad y Electr6nica at the University of Seville, where he became Assis- tant Professor in 1973 and a Professor in Elec- tronics in 1981. His research interest lies in the field of multivalued logic, sequential machines, analog circuit design, and nonlinear network analy- sis and synthesis. Angel Rodriguez-Vbquez (M80) received the Licenciado en Fisica and the Doctor en Ciencias Fisicas degrees from the University of Seville, Spain, in 1977 and 1983, respectively. Authorized licensed use limited to: Universidad de Sevilla. Downloaded on April 01,2020 at 16:04:47 UTC from IEEE Xplore.  Restrictions apply. 

IEEE Transactions on Circuits and Systems

A Programmable Neural Oscillator Cell

https://idus.us.es/bitstream/handle/11441/94806/A%20Programmable%20Neural%20Oscillator%20Cell.pdf?sequence=1&isAllowed=y

A Programmable Neural Oscillator Cell

Abstract

Similar works

Full text

Available Versions

idUS. Depósito de Investigación Universidad de Sevilla