Contains a report on a research project.Bell Telephone Laboratories, IncorporatedTeagle FoundationNational Science Foundatio

McCulloch, Warren S.

DSpace@MIT

XVI. NEUROPHYSIOLOGYW. S. McCulloch R. C. Gesteland W. H. PittsE. M. Duchane J. Y. Lettvin P. D. WallA. THREE OF VON NEUMANN'S BIOLOGICAL QUESTIONSNeurophysiologists are indebted to John von Neumann for his studies of componentsand connections in accounting for the steadiness and the flexibility of behavior. Inspeaking to the American Psychiatric Association (1) he stressed the utility and theinadequacy of known mechanisms for stabilizing nervous activity, namely, (a) the thresh-old of nonlinear components, (b) the negative feedback of reflexive mechanisms, (c)the internal switching to counteract changes-"ultrastability"-(2), and (d) the redundancyof code and of channel. He suggested that the flexibility might depend upon local shiftsof thresholds or incoming signals to components that are more appropriate to computersthan any yet invented. His Theory of Games (3) has initiated studies that may discloseseveral kinds of stability and has indicated where to lookfor logical stability under common shift of threshold.BOTH His "Toward a Probabilistic Logic" (4) states the prob-A ALONE 6ALONE lem of securing reliable performance from unreliableNEITHER components, but his solution requires better relays thanthan he could expect in brains. These, his interests,X x > x. -x. - X-* put the questions we propose to answer. His satisfaction.Y \ ".<- . "~" with our mechanisms for realizing existential and uni-versal quantification in nets of relays (5, 6) limits ourA/B=A.XaB Ave'-AXBtask to the finite calculus of propositions. Its perform-- A -A A)B=A XBance has been facilitated by avoiding the opacity of theA.B=A B A=B'- Bfamiliar symbols of logic and the misleading suggestionsA/(B/B) BD A A * Eof multiplication and addition modulo two of the facile(A/(B/B))/(A/(B/B))= A.- B.V.B.-A= A.X . B boolean notation for an algebra that is really substitutive((A/A)/(B/B))/((A/A)/(B/B)) = -A- -B z A X B (7, 8, 9). Our symbols have proved useful in teachingFig. XVI-1. Symbols. symbolic logic in psychological and neurological con-texts (10). Familiarity with them undoubtedly contrib-uted to the invention of the circuits whose redundancy permits solution of our problems.See Fig. XVI-1.The finite calculus of propositions can be written at great length by repetitionsof a stroke signifying the incompatibility of its two arguments. The traditional fivesymbols, for 'not', 'both', 'or', 'implies', and 'if and only if', shorten the textThis work was supported in part by Bell Telephone Laboratories, Incorporated;in part by The Teagle Foundation, Incorporated; and in part by National Science Foun-dation.129(XVI. NEUROPHYSIOLOGY)but require conventions and rearrangements in order to avoid ambiguities. Economyrequires one symbol for each of the sixteen logical functions of two propositions. Theonly necessary convention is then one of position or punctuation.Since the logical probability and the truth value of a propositional function are deter-mined by its truth table, each symbol should picture its table. When the place in thetable is given, any jot serves for "true" and a blank for "false." When the four placesin the binary table are indicated by a cross (X) it is best to let the place to the left showthat the first proposition alone is the case; to the right, the second; above, both; andbelow, neither. Every function is then pictured by jots for all of those cases in whichthe function is true, ranging from contradiction, with no jots, to tautology, with four.Formulas composed of our symbols are transparent when the first proposition isrepresented by a letter to the left of the symbol and the second to the right. When thesespaces are occupied by logical variables the formula is that of a propositional function;when they are occupied by propositions, of a proposition; consequently the formula canoccupy the position of an argument in any subsequent formula.Two distinct propositions, A and B, are independent when the truth value of eitherdoes not logically determine the truth value of the other. A formula with only one symbolwhose spaces are occupied by two independent propositions can never have the symbolwith no jots or four jots. The truth value of any other function is contingent upon thetruth value of its arguments. Let us call it "a significant proposition of the first rank."A formula for a proposition of the second rank is formed by inserting in the spacesof its symbol two significant propositions of the first rank; for example, (A .X B) > (A X. B).When the two propositions of the first rank are composed of the same pair of propositionsin the same order, the resulting function of the second rank can always be equated to oneof the first rank; for example, (A *. B) < (A . B) = (A -X B), by putting jots into the crossby the four rules of reduction:1) When the central X has a jot at the left, insert a jot in the new X for every jotin the X to the left but not in the X to the right. (A . B)X (A X B) = (A X. B).2) When the central X has a jot at the right, insert a jot in the new X for every jotin the X to the right but not in the X to the left. (A X. B) X (A .X B) = (A .x B).3) When the central X has a jot above, put a jot in the new X for all jots commonto the right and left X's. (A -X B) (A XB) = (A X B).4) When the central X has a jot below, put a jot in the new X for every space emptyin both the right and left X's. (A .X B) X (A X. B) = (A ; B).By repetition of the construction we can produce formulas for functions of the thirdand higher ranks and reduce them step by step to the first rank, thus discovering theirtruth values.Since in what follows no other formulas are necessary, the letters A and B will beomitted, and positions, left and right, will replace parentheses.130(XVI. NEUROPHYSIOLOGY)In formulas for significant functions of the first rank the chance addition or omissionof a jot produces an erroneous formula and will cause an error only in that case forwhich the jot is added or omitted, which is one out of the four logically equiprobablecases. With similar symbols for functions of three arguments, the error will occur inonly one of the eight cases, and, in general, for functions of 6 arguments, in one of 2cases. If p is the probability of the erroneous jot and P the probability of error pro--6duced, P = 2 p.In formulas for the second rank there are three symbols. If we relax the requirementof independence of the arguments, A and B, there are then 163 possible formulas eachof which reduces to a formula of the first rank. Thus the redundancy, R, of these for-mulas of the second rank is 163 - 16 = 162. For functions of 6 arguments, R = (226)6To exploit this redundancy so as to increase the reliability of inferences from unre-liable symbols let us realize the formulas in nets of what von Neumann called neurons(3). Each neuron is a relay which on receipt of all-or-none signals either emits anall-or-none signal or else does not emit one which it would otherwise have emitted.Signals approaching a neuron from two sources either do not interact, or those fromone source prevent some or all of those from the other source from reaching the recip-ient neuron. The diagrams of the nets of Fig. XVI-2 are suggested by the anatomy ofthe brain. They are to be interpreted as follows.A line terminating upon a neuron shows that it excites it with a value +1 for eachtermination. A line forming a loop at the top of the neuron inhibits it with a value ofexcitation of -1 for each loop. A line forming a loop around a line approaching a neu-ron shows that it prevents excitation or inhibition from reaching the neuron throughthat line.Each neuron has on any occasion a threshold, 0, measured in steps of excitation,and it emits a signal when the excitation it receives is equal to or greater than 0. Theoutput of the neuron is thus some function of its input, and which function it is dependsupon both its local connections and the threshold of the neuron. These functions can besymbolized by crosses and jots beginning with none and adding one at a time as 0decreases until all four have appeared in the sequence noted in the legend for its diagramin Fig. XVI-2. Because all 24 sequences (of which only 12 left-handed are drawn) arethus realized we can interpret the accidental gain or loss of a jot or jots in an intendedsymbol as a change in the threshold of an appropriate neuron.The formula for a function of the second rank is realized by a net of three neuronseach of whose threshold is specified; for example, see Fig. XVI-3. The function canbe reduced to one of the first rank whose symbol pictures the relation of the output ofthe net to the input of the net.When all thresholds shift up or down together, so that each neuron is representedby one more, or one less, jot in its symbol but the reduced function is unaltered, the13194 X32- I-2-3LEGEND:1 22X3 'X44 384 X3 *X2 Xo *.2LEGEND-3LEGEND. 1X34 X3 .X X2 -X" -X•4<* -x.0o .*LEGEND: IX24Fig. XVI-2. Diagrams.132TxX.2X47r~ 7r7T T42X3I8 -I 0 +18 -2 -1 08 -3 -2 -1Fig. XVI-3. A logically stable net.REDUCED SYMBOL ERRORSCASE PROBABILITY(I-p) p(I-p)(I-p) p(p)P2= 2 (I-p)pFig. XVI-4. A best stable net.133CONDITIONxv ;X.).* X. .X.- >"-<. ; -X.*- X.X.- X.x.- x.?- ;."(XVI. NEUROPHYSIOLOGY)net is called "logically stable."The redundancy of functions of the second rank provides us with many examples ofpairs of functions and even triples of functions that reduce to the same function of thefirst rank and that can be made from one another by common addition or omission ofjots in all symbols, and the diagrams of Fig. XVI-2 enable us to realize them all in sev-eral ways: For example, there are 32 triples of functions and 64 logically stable netsfor every reduced formula with a single jot.If such nets are embodied in our brains they answer von Neumann's repeated questionof how it is possible to think and to speak correctly after taking enough absinthe or alco-hol to alter the threshold of every neuron. The limits are clearly convulsion and coma,for no function remains significant or stable with a shift of 0 that reduces the outputneuron to tautology or to contradiction. The net of Fig. XVI-3 is logically stable overthe whole range between these limits. Let the causes and probabilities of such shiftsbe what they may, those that occur simultaneously throughout these nets create no errors.Logically stable nets differ greatly from one another in the number of errors theyproduce when thresholds shift at random in their neurons and the most reliable makesome errors; for example, the net of Fig. XVI-4.To include random shifts, let our symbols be modified by replacing a jot with 1 whenthe jot is fixed and with p when that jot occurs with a probability p, and examine theerrors extensively as in Fig. XVI-4. Here we see that the frequency of the erroneousformulas is p (1 - p), and the actual error is a deficit of a jot at the left in each faultystate of the net, i.e., in one case each. Hence we may write for the best of stable netsP 2-2P2 = 2p (1 - p). The factors p and (1 - p) are to be expected in the errors of any netwhich is logically stable, for the errors are zero when p = 0 or p = 1. No stable net ismore reliable.No designer of a computing machine would specify a neuron that was wrong more thanhalf of the time; for he would regard it as some other neuron wrong less than half of thetime; but in these most useful of logically stable circuits, it makes no difference whichway he regards it, for they are symmetrical in p and 1 - p. At p = 1/2, the fre--2quency of errors is maximal and is P2 =  2 1/2(1 - 1/2) = 1/16, which is twice as reli--2able as its component neurons for which P1 = 2 1/2 = 1/8.Among logically unstable circuits the most reliable can be constructed to securefewer errors than the stable whenever p < 1/2. The best are like that of Fig. XVI-5.The errors here are concentrated in the two least frequent states and in only one of the-2 2four cases. Hence P2 - 2 p .When 6 = 3, the redundancy, R =- (22 , provides so many more best stable andbest unstable nets that the numbers become unwieldy. There are (22 5)+1 nets for func-tions of the second rank each made of 4 neurons to be selected from 8 ! diagrams with9 thresholds apiece. Yet it is clear that the best stable and unstable for 5 > 2 are better134XP= 3P:= 2 PERRORSCASE PROBABILITY*)( (p)p(l-p)-X (Pp)p(p)Fig. XVI-5.p0=3AA best unstable net.0=3BFig. XVI-6. Unstable improvement net.135PX8 3S= 2- pCONDITION REDUCED SYMBOL< .< ;(x *x( fx%< >t P,.•; )< % %• .; >i• *k .• < .< <P? = 2 p 2< P, =2Vp(XVI. NEUROPHYSIOLOGY)-6than those for 6 = 2 only in the factor 2 for error in a single case.Further improvement requires the construction of nets to repeat the improvementof the first net and, for economy, the number of neurons should be a minimum. Forfunctions of 6 arguments each neuron has inputs from 6 neurons. Hence the width ofany rank is 6, except the last, or out-A B put, neuron. If n be the number ofranks, then the number of neurons, N,is 6 (n- i) + 1.k UNSTABLE (p Figure XVI-6 shows how to construct8:3 2" p 8=3p 0 one of the best possible nets for theunstable ways of securing improvementSTABLE with two output neurons as inputs for-I 2 p(-p) - the next rank. The formulas are selectedto exclude common errors in the outputneurons on any occasion. In these, theP ( / UNSTABLE / XP best of unstable nets, the errors of theoutput neurons are P = 2 p .[Whether we are interested in shiftsof threshold or in noisy signals it isSTABLE 2 proper to ask what improvement is to beexpected when two or three extra jotsappear in the symbols. With our neu-rons a second extra jot appears only ifPX UNSTABLE9= 3 p=2Pp3(,-p,2 3 the first has appeared, and a third onlyif the second. If the probability p ofn EVEN,P n  pn/2 (I- n/ 2an extra jot is kept constant, the pro-2Fig. XVI-7. Alternating improvement net. bability of two extra jots is p and ofthree is p . Examination of the net inFig. XVI-6 shows that P2 < 1' ifp + p + p < 0.15 or p < 0.13. Tomatch Gaussian noise the log of successive p's should decrease as (AO)2, or 1,4 9p, p , p , giving P 2 <P 1 for p < 0.25. The remaining errors are always so scatteredas to preclude further improvement in any subsequent rank.]When common shifts of 0 are to be expected, or all we know is 0 < p < 1, a greaterimprovement is obtained by alternating stable and unstable nets as in Fig. XVI-7,selected to exclude common errors in its output neurons. The improvement for n even isP = 2-6 pn/2 (I p)n/2and the errors to be expected are1360m.5< .><. .5<0 x xN - 0 -to.xd0  +5< 5< 5<C I 00t 0 .<C ID o -*..% *X* x * 4* t*x . . . *x 4 - .cl i . * c-i -- .NXN ,N N•.. .. . x .<.. x. .. < *x x x x. <.x 4 < .%<. x 5< .4< x0 o N o - o ox .* X. X. X. X. X X. X.z 0 0 0 0 0 0 0 00< o 0 0 0 . -0>t .< .5< .<. .5<.X. XA .. .X. X.o 0oC ? $. . $. .N N N N NN NNXN~137NOx c(XVI. NEUROPHYSIOLOGY)1 n2-6 pn/2 )n/2p 2- (n+1)!which is less than2n-6 (n) n + 1(n+ 1)! nand hence a minimum for any proportion of stable and unstable nets.None of these nets increases reliability in successive ranks under von Neumann'scondition that neurons fire or fail with probability p regardless of input; but they aremore realistic and more interesting neurons.Despite the increase in reliability of a net for a function of the second rank thesenets can be compelled to compute as many as 13 of the 16 reduced functions by alteringthe thresholds of the neurons by signals from other parts of the net, as in Fig. XVI-8,and even logically stable nets for triples of functions can be made to compute 8 of the 16,as in Fig. XVI-9.Thus these appropriate neurons have served our purpose in answering the questionsof logical stability under common shift of threshold and of securing reliable performancefrom unreliable components without losing the flexibility of functions computed by netscomposed of them.W. S. McCullochReferences1. J. von Neumann, Fixed and random logical patterns and the problem of reliability,American Psychiatric Association, Atlantic City, May 12, 1955.2. W. Ross Ashby, Design For a Brain (John Wiley and Sons, Inc., New York,Ist Ed., 1952).3. J. von Neumann and 0. Morgenstern, Theory of Games and Economic Behavior(Princeton University Press, Princeton, N.J., 1st Ed., 1944).4. J. von Neumann, Probabilistic Logics and the Synthesis of Reliable OrganismsFrom Unreliable Components (Automata Studies, Edited by C. E. Shannon andJ. McCarthy, Princeton University Press, Princeton, N.J., 1956).5. W. S. McCulloch and W. Pitts, A logical calculus of the ideas immanent in nervousactivity, Bull. Math. and Biophys. 5, 115 (1943).6. W. Pitts and W. S. McCulloch, How we know universals. The perception of audi-tory and visual forms, Bull. Math. and Biophys. 9, 127 (1947).7. K. Menger, Analysis without variables, J. Symbol. Logic 11, No. 1, 30-31 (1946).8. K. Menger, Algebra of Analysis (Notre Dame Mathematical Lectures, No. 3,University of Notre Dame, Notre Dame, Ind., 1944).9. K. Menger, General algebra of analysis, Reports of a Mathematical Colloquium,Series 2, issue 7, p. 46 (1946).10. W. S. McCulloch, Machines That Know and Want (Brain and Behavior, A Symposium,edited by W. C. Halstead; Comparative Psychology Monographs 20: No. 1: p. 39,University of California Press, Berkeley and Los Angeles, 1950).138

Neurophysiology

http://dspace.mit.edu/bitstream/1721.1/52115/1/RLE_QPR_047_XVI.pdf

Neurophysiology

Abstract

Similar works

Full text

Available Versions

DSpace@MIT

DSpace@MIT