The aim is to obtain computationally more powerful, neuro physiologically founded, artiﬁcial neurons and neural nets.  Artiﬁcial Neural Nets (ANN) of the Perceptron type evolved from the original proposal by McCulloch an Pitts classical paper [1]. Essentially, they keep the computing structure of a linear machine followed by a non linear operation. The McCulloch-Pitts formal neuron (which was never considered by the author’s to be models of real neurons) consists of the simplest case of a linear computation of the inputs followed by a threshold. Networks of one layer cannot compute anylogical function of the inputs, but only those which are linearly separable. Thus, the simple exclusive OR (contrast detector) function of two inputs requires two layers of formal neuron

Blasio, Gabriel de

Moreno Díaz, Arminda

Moreno Díaz, Roberto

English

Archivo Digital UPM

A Framework for Modelling Competitive and Cooperative Computation in Retinal Processing.

A Logical Calculus of the Ideas Immanent in Nervous Activity,

Contrast Detection in Frog’s Retina.

Deterministic and Probabilistic Neural Nets with Loops.

Orientation Selectivity in Rabbit Retinal Ganglion Cells is Mediated by Presynaptic Inhibition.

Properties of a Neuron with Many Inputs. In: Principles of Self organitation.

Synaptic Computation.

The Fun of Failures.

Artificial Neural Nets with Interaction of Afferents

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Artiﬁcial Neural Nets with
Interaction of Aﬀerents
Gabriel de Blasio1, Arminda Moreno-Dı´az2, Roberto Moreno-Dı´az1
1 Instituto Universitario de Ciencias y Tecnolog´ıas Ciberne´ticas
Universidad de Las Palmas de Gran Canaria
gdeblasio@dis.ulpgc.es
rmoreno@ciber.ulpgc.es
2 School of Computer Science. Madrid Technical University
amoreno@fi.upm.es
Extended Abstract
The aim is to obtain computationally more powerful, neurophysiologically
founded, artiﬁcial neurons and neural nets.
Artiﬁcial Neural Nets (ANN) of the Perceptron type evolved from the original
proposal by McCulloch an Pitts classical paper [1]. Essentially, they keep the
computing structure of a linear machine followed by a non linear operation. The
McCulloch-Pitts formal neuron (which was never considered by the authors to
be models of real neurons) consists of the simplest case of a linear computation
of the inputs followed by a threshold. Networks of one layer cannot compute any
logical function of the inputs, but only those which are linearly separable. Thus,
the simple exclusive OR (contrast detector) function of two inputs requires two
layers of formal neurons.
Those logical limitations where overcome by McCulloch and Blum [2] by
a formalization of the, by then recently encountered, presynaptic inhibition in
Rana Pipiens. In essence, ﬁbres reaching a neuron bifurcate in a way that they
may drastically inhibit other input ﬁbers to the neuron. Posterior and more
recent ﬁndings emphasize the importance and role of presynaptic inhibition and
facilitation in the complexity of neuronal computation [3], [4], [5].
The systematic formulation of presynaptic inhibition, for logical functions,
consists in substituting the simple linear weighted addition prior to the non-
linear operation given by the threshold function. Thus, the typical weighted
summing computation (for x1, . . . , xn lines)
∑
αixi is, in general, substituted
by the more complete (redundant) operation
∑
i
αixi +
∑
ij
αijxix¯j +
∑
ijk
αijkxix¯j x¯k + · · · (1)
where x¯i denotes logical negation; xix¯j denotes presynaptic inhibition of ﬁber
xi by ﬁber xj ; xix¯j x¯k . . . denotes the presynaptic inhibition of xi by xj , xk,
and so on. Figure 1 illustrates equation 1 where small ’loops’ indicate inhibition
presynaptic to the neuron. In formal neural nets, the inhibition is total, that is,
the ’loop’ completely inhibits the input signal xi.
62
xi
xj
xk
i
ij
ijk
Fig. 1. Illustration of general lateral inhibition interaction of aﬀerents for McCulloch-
Pitts neurons.
For a single unit with a ﬁxed threshold, the number of degrees of freedom
for M inputs lines, is
M + M(M − 1) + M
(
M − 1
2
)
+ · · ·+
(
M − 1
M − 2
)
+ M = M · 2M−1
which is larger than the number of possible functions for M inputs, 2M , and
which clearly points to the redundances eﬀects provoked by the presynaptic
inhibition.
It can be argued that this redundancy can be used to increase reliability for a
net computing any arbitrary logical function. In fact, there is a trade oﬀ between
reliability and versatility provoked by the presynaptic inhibition.
We propose a natural generalization of the logical formulations of presynaptic
inputs inhibitory interaction, which allows for a richer model of formal (artiﬁ-
cial) neuron, whose potentialities as computing units are to be investigated. The
formulation reduces to classical interaction of aﬀerents for logical inputs.
First, normalize input signals so that 0 ≤ xi ≤ 1. The presynaptic inhibition
of signal xi by signal in ﬁber j, xj is given by a multiplicative eﬀect xi(1− xj).
Similarly, for the rest of the ﬁbers.
Thus, the argument of the activation function of the corresponding artiﬁcial
neuron, is
A =
∑
i
αixi +
∑
ij
αijxi(1− xj) +
∑
ijk
αijkxi(1− xj)(1− xk) (2)
If F is the activation function, the output, y, of the AN is
y(k + 1) = F [A(k)] discrete time
dy
dt
= y˙ = F [A(t)] continuous time
63
x
1 x2
4
-1
2
(a)

x
1
x
2
(b)
Fig. 2. Examples of two realizations for an AN computing the ’exclusive OR’. a) Neu-
ron in this ﬁgure is more versatile in the sense that relatively small changes in the
activation function (e.g. a thresholding ) provokes signiﬁcant changes in the discrimi-
nating behaviour. b) Neuron in this ﬁgure is more insensitive to threshold changes.
In general, for an ANN with arbitrary feedback, the expressions are similar
than those of formal neural nets [7], [8]. In the present case, the networks are
given by
yi(k + 1) = F [A(x1(k) . . . xM (k); y1(k) . . . yN (k))] i = 1, 2, . . . , N
or
y˙(t) = F [A(x1(t) . . . xM (t); y1(t) . . . yN (t))]
The activation function F is assumed to be the same for all neurons of the
net. Essentially, the new formulation substitutes the linear activation argument
of typical ANN (Hopﬁeld type) formulations for a neurophysiological plausible
non-linear argument, result of presynaptic interaction.
As preliminary results, various examples are presented of non linear acti-
vation arguments and their input pattern discriminatory power, their reliability
and their functional versatility (range of functions that can be computed by each
AN). This is illustrated for the case of an AN of two inputs, in Figure 2. Fig-
ures 2(a) and 2(b) are two realizations for the ’exclusive OR’ (contrast detection).
Neuron in ﬁgure 2(a) is more ’versatile’ in the sense that relatively small changes
in the activation function (e.g. a thresholding), provokes signiﬁcant changes in
the discriminating behaviour. Neuron in ﬁgure 2(b) is more insensitive to thresh-
old changes. The corresponding activation arguments are shown in Figure 3(a)
and 3(b).
64
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
0
0.5
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(a)
0 0.2 0.4 0.6 0.8 1 0
0.2
0.4
0.6
0.8
1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
(b)
Fig. 3. Activation arguments of AN in Figure 2(a) and 2(b). Notice that the patterns
[(only x1)] and [(only x2)] are better discriminated by activation argument in ﬁgure 3(b)
References
1. McCulloch, W.S., Pitts, W.H. A Logical Calculus of the Ideas Immanent in Nervous
Activity, Bulletin of Mathematical Byophysics, 5, 115–133 (1943)
2. Blum, M. Properties of a Neuron with Many Inputs. In: Principles of Self organita-
tion. Von Foerster, Zopf, R. (eds.). Pergamon Press, New York, 95–119 (1961)
3. Schipperheyn, J.J. Contrast Detection in Frog’s Retina. Acta Physiol. Pharmacol.
Neerlandica 13, 231–277 (1965)
4. Abbott, L.F., Regehr, W.G. Synaptic Computation. Nature, 431 796–803 (2004)
5. Venkataramani, S., Taylor, W. R. Orientation Selectivity in Rabbit Retinal Ganglion
Cells is Mediated by Presynaptic Inhibition. The Journal of Neuroscience, 30 (46),
15664–15676 (2010)
6. McCulloch, W.S., Papert, S.A., Blum, M., da Fonseca, J.S., Moreno-Dı´az, R. The
Fun of Failures. Annals of the New York Academy of Sciences, 156 (2), 963–968
(1969)
7. Moreno-Dı´az, R. Deterministic and Probabilistic Neural Nets with Loops. Math.
Biosciences, 11 129–131 (1971)
8. Moreno-Dı´az, R., de Blasio, G., Moreno-Dı´az, A. A Framework for Modelling Com-
petitive and Cooperative Computation in Retinal Processing. In Collective Dynam-
ics: Topics on Competition and Cooperation in the Biosciences. L. M. Ricciardi,
Buonocuore, A. and Pirozzi, E., eds. American Institute of Physics, 88–97 (2008)
65


Artificial Neural Nets with Interaction of Afferents

Abstract

Similar works

Full text

Available Versions

Archivo Digital UPM

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid