An experiment using the phenomenon of percolation has been conducted to demonstrate the implementation of neural functionality (summing and sigmoid transfer). A simple analog approximation to digital percolation is implemented. The device consists of a piece of amorphous silicon with stochastic bit-stream optical inputs, in which a current percolating from one end to the other defines the neuron output, also in the form of a stochastic bit stream. Preliminary experimental results are presented

Daalen, M.

Hall, T.J.

Hands, M.A.

Parmiter, P.J.M.

Pignon, D.

Shawe-Taylor, J.

Slack, J.K.

English

Southampton (e-Prints Soton)

222 OPTICS LETTERS / Vol. 21, No. 3 / February 1, 1996Sigmoid neural transfer function realized by percolationD. Pignon, P. J. M. Parmiter, J. K. Slack, M. A. Hands, and T. J. HallKing’s College, University of London, Strand, London WC2R 2LS, UKM. van Daalen and J. Shawe-TaylorRoyal Holloway College, University of London, Egham, Surrey TW20 0EX, UKReceived September 12, 1995An experiment using the phenomenon of percolation has been conducted to demonstrate the implementation ofneural functionality (summing and sigmoid transfer). A simple analog approximation to digital percolation isimplemented. The device consists of a piece of amorphous silicon with stochastic bit-stream optical inputs, inwhich a current percolating from one end to the other defines the neuron output, also in the form of a stochasticbit stream. Preliminary experimental results are presented. ã 1996 Optical Society of AmericaCurrent interest in digital stochastic bit-streamrepresentation of information for neural processingsystems1 opens up the possibility of using a simpleanalog percolation device to implement neural func-tionality. This is possible because of the emergentresponse of a finite percolation grid2 to a matrix ofstochastic inputs, which in turn renders such a devicesuitable for use in a free-space optoelectronic system.3,4One can use a continuous stream of digital bitsto represent an analog value by modulating thebits stochastically with a probability related to thevalue to be encoded. This results in a bit stream inwhich the proportion of 1’s gives the probability p,where p ­ x for an analog value x in the interval(0, 1) and p ­ sx 1 1dy2 for x in the interval s21, 1).The advantage of such a representation is that simpledigital hardware can be used, with the good errorperformance of digital systems combined with thefunctionality of analog processing.1 For instance,multiplication of two bit streams—such asweight 3 input for a neuron—reduces to a simplebitwise XNOR [for the s21, 1d interval, or AND for the(0, 1) interval] of successive bit pairs from the twostreams, producing a single bit stream representingthe product of the two values.In the context of an optoelectronic implementa-tion3,4 the high available bandwidth inherent in opticalbeamlet channels permits very high (gigahertz) bit-stream modulation rates.The focus of the study presented here is to producea simple device to perform the summing and thresh-olding functions of a neuron.5 We are working on athree-dimensional optoelectronic implementation of aneural network6 that uses planes of simple electronicdevices for processing, densely interconnected by opti-cal beamlet channels. Bearing in mind such a three-dimensional optoelectronic implementation, what isrequired is a planar device to receive a two-dimensionalgrid of optical bit-stream inputs (already multipled bytheir corresponding weights by an array of optoelec-tonic XNOR gates in the preceding plane), to sum them,and then to apply a sigmoid transfer function to thesum, thus providing the single analog output from theneuron in stochastic bit-stream form.This implementation may be simply realizedby a two-dimensional network of percolating cells(Fig. 1).2 Each input is incident upon a single cell inthe grid (of N 3 N cells) that then either conductsor not, depending on whether the incident optical bitis high or low. Each cell is connected to its nearestneighbors (four connected) or further [eight connected(see Fig. 2) etc.], and a voltage is applied across thetop and bottom rows of cells. At any one instant thecollective bits from the incident streams may or maynot cause a conducting path between the contacts. Ifthe current flowing between the contacts is interpreteddigitally, i.e., no current ­ 0 and some current ­ 1,another stochastic bit stream results that representsthe sum of the inputs with a sigmoidlike transferfunction applied, as desired. The sigmoidlike shape ofthe transfer function appears as an emergent propertyof percolation through a finite grid.7 The smaller thegrid, the shallower the curve, and in the limit of aninfinite grid the curve becomes a step. Also, a higherconnectivity makes the curve steeper. Figure 3shows a computer simulation of the bit-stream digitalpercolation grid.We report on an experiment to assess the feasi-bility of further simplification of the device by theuse of an analog approximation. This consists of atwo-dimensional piece of amorphous silicon with theoptical bit-stream inputs incident upon the face andcontacts appended on two sides as described aboveFig. 1. Digital percolation grid with bit-stream inputs andoutputs.0146-9592/96/030222-03$6.00/0 ã 1996 Optical Society of AmericaFebruary 1, 1996 / Vol. 21, No. 3 / OPTICS LETTERS 223Fig. 2. Example of an eight-connected cell.Fig. 3. Computer simulation of a digital percolation grid.16 3 16, 8 connected, 100 kbits per input.(Fig. 4). Each cell is now defined by the input beam-lets themselves, where a high bit (high intensity) pro-duces photogenerated charge carriers in its cell (of re-sistivity Ron) and a low bit (low intensity) leaves a cellof high resistivity, Roff. A rough calculation suggeststhat for this device to be equivalent to the digital onedescribed above the necessary optical contrast ratio be-tween high and low states is given byRoffyRon $ NsN 2 2d 1 1,with the absolute optical power in each channel scalingin a similar manner because of the need to overcomebackground conductivity of the silicon.In the experiment we produced the bit streams bymodulating an expanded laser beam, using a ferroelec-tric liquid-crystal spatial light modulator (SLM). Theinput beam was chopped, and a lock-in amplifier wasused to measure the current across the device. A com-puter was used to produce the bit streams on theSLM and to gather the data. After a threshold cur-rent was determined by means of two test patterns, acurrent above this threshold produced a 1 output bit,and a current below produced a 0. We attached probewires to the amorphous silicon wafer, using silver DAGpaint, creating ohmic, rather than Shottky as would beexpected from a clean metal–semiconductor junction,contacts, as required.Initially we used a 5 3 5 grid to obtain a shallowcurve and also because, as mentioned above, this wouldrequire a contrast ratio of only 16:1 between the highand the low signals to be equivalent to the digitalpercolation as simulated on the computer. However,with the particular setup used, because of leakagebetween SLM pixels and reflections, the contrast ratioobtained was 14:1.Nevertheless, preliminary results are highly encour-aging, as shown by Fig. 5. The curve was found to bealmost exactly described by the sigmoid formy ­ 1yf1 1 expsax 1 bdg,with coefficients a and b equal to 220.0 and 8.9,respectively. One can translate the curve along thex axis (b modified) at will by varying the currentthreshold.At a contrast ratio of 16:1 or greater the deviceoperates as a digital percolation matrix with a con-nectivity that depends on the contrast ratio; the lowerthe ratio, the more effect a current through a distantcell will have on the cell in question, increasing theeffective connectivity. At a ratio below this value (asindeed was the case with the preliminary runs) a sec-ond, analog, effect begins to affect the result. Thisis the position-independent sum of cell resistivities.In the experiment the summing effect alone wouldFig. 4. Schematic of the analog percolation experiment.For clarity, the imaging optics are not shown.Fig. 5. Experimental result. 5 3 5 grid, 1500 bits, 60steps.224 OPTICS LETTERS / Vol. 21, No. 3 / February 1, 1996produce a step. In reality there is a combination ofsumming and percolation, the relative importance ofeach being determined by contrast ratio. The effectof the summing on the percolation graph is to makeit steeper.In summary, we have shown that an analo opto-electronic percolation grid can be used to performthe sigmoid transfer function required by a neuron.Even from a preliminary nonoptimized experimentthe result obtained was highly encouraging. Furtherquick improvements under way, such as the intro-duction of a pixelating computer-generated hologram,should result in an improvement of contrast ratio to100:1, permitting a percolation grid of 11 3 11 oreven more cells. Methods for determining the rateof change of the output with respect to each input al-ready exist for the perfect digital case, so these willalso be investigated. Gradient information is helpfulto neural network training algorithms.The advantage of such a device is its great ease offabrication and low cost, because its natural emergentproperties are exploited without the need for fabrica-tion of complex devices.The research of D. Pignon is supported by anEngineering and Physical Sciences Research CouncilPh.D. Cooperative Award in Science and Engineeringcosponsored by GEC-Marconi. The authors thankBill Crossland and Steven Latham, then at Cam-bridge University, for supplying the amorphous siliconwafer for use in this experiment, and Pilkingtons,British Aerospace, and GEC for providing otherequipment.References1. J. Shawe-Taylor, P. Jeavons, and M. van Daalen, Con-nect. Sci. 3, 317 (1991).2. T. J. Hall and M. van Daalen, UK patent application9420449.2 (October 11, 1994).3. D. Pignon and T. J. Hall, ‘‘A quantised optical neuralnetwork with on-line learning,’’ submitted to Int. J.Optoelectron.4. A. G. Kirk, S. Jamieson, H. Imam, and T. J. Hall, Opt.Comput. Process. 2, 293 (1992).5. P. K. Simpson, Artificial Neural Systems (Pergamon,Oxford, 1990).6. T. J. Hall, W. Peiffer, M. Hands, H. Thienpont,W. A. Crossland, J. S. Shawe-Taylor, and M. van Daalen,in Optical Computing, Vol. 10 of 1995 OSA TechnicalDigest Series (Optical Society of America, WashingtonD.C., 1995), p. 210.7. D. Stauffer and A. Aharony, Introduction to PercolationTheory (Taylor & Francis, London, 1992).

A quantised optical neural network with on-line learning,’’ submitted to Int.

Artificial Neural Systems (Pergamon,

Introduction to Percolation Theory

Sigmoid Neural Transfer Function Realised by Percolation

Sigmoid Neural Transfer Function Realised by Percolation

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)