Furber, S.B.

Kawasetsu, T.

Okuno, H.

Plana, Luis A.

Yagi, T.

English

To understand the functional roles of visual neurons in the retina and the visual cortex, responses of a visual neuronal network under natural visual environments should be investigated. In this study, we developed an emulation platform for reproducing neural activities in the retina and the visual cortex with the following features: real-time reproduction of neural activities with physiologically feasible spatio-temporal properties and configurable model structure. To achieve both real-time simulation and configurability, we employed a mixed analog-digital architecture with multiple parallel processing techniques. The system was composed of a silicon retina with analog resistive networks, a field-programmable gate array, and SpiNNaker chips. The emulation system was successful in simulating a part of the retinal and cortical circuits at 200 Hz

The University of Manchester - Institutional Repository

The University of Manchester ResearchA real-time simulator of a biological visual systemcomposed of a silicon retina and SpiNNaker chipsLink to publication record in Manchester Research ExplorerCitation for published version (APA):Okuno, H., Kawasetsu, T., Plana, L. A., Furber, S. B., & Yagi, T. (2014). A real-time simulator of a biological visualsystem composed of a silicon retina and SpiNNaker chips. In 2014 International Symposium on Artificial Life andRobotics (AROB 2014)Published in:2014 International Symposium on Artificial Life and Robotics (AROB 2014)Citing this paperPlease note that where the full-text provided on Manchester Research Explorer is the Author Accepted Manuscriptor Proof version this may differ from the final Published version. If citing, it is advised that you check and use thepublisher's definitive version.General rightsCopyright and moral rights for the publications made accessible in the Research Explorer are retained by theauthors and/or other copyright owners and it is a condition of accessing publications that users recognise andabide by the legal requirements associated with these rights.Takedown policyIf you believe that this document breaches copyright please refer to the University of Manchester’s TakedownProcedures [http://man.ac.uk/04Y6Bo] or contact uml.scholarlycommunications@manchester.ac.uk providingrelevant details, so we can investigate your claim.Download date:09. Jun. 2022A real-time simulator of a biological visual system composed of a silicon retina and SpiNNaker chips   Hirotsugu Okuno1, Takumi Kawasetsu1, Luis A. Plana2, Steve B. Furber2, and Tetsuya Yagi1 1Osaka University, Japan 2The University of Manchester, United Kingdom (Tel: 81-6-6879-7786, Fax: 81-6-6879-7784)  1h-okuno@eei.eng.osaka-u.ac.jp  Abstract: To understand the functional roles of visual neurons in the retina and the visual cortex, responses of a visual neuronal network under natural visual environments should be investigated. In this study, we developed an emulation platform for reproducing neural activities in the retina and the visual cortex with the following features: real-time reproduction of neural activities with physiologically feasible spatio-temporal properties and configurable model structure. To achieve both real-time simulation and configurability, we employed a mixed analog-digital architecture with multiple parallel processing techniques. The system was composed of a silicon retina with analog resistive networks, a field-programmable gate array, and SpiNNaker chips. The emulation system was successful in simulating a part of the retinal and cortical circuits at 200 Hz.  Keywords: silicon retina, SpiNNaker, retina, visual cortex, simulation  1 INTRODUCTION Visual signals are received and processed in the retina and the visual cortex. For understanding the computation and function of these biological visual systems, the relationship between visual inputs and neural responses have been investigated by using simple visual stimuli, e.g., spots, annuli, and gratings of light in physiological experiments for several decades [1][2]. Various neuronal network models of the retina and the cortex have been proposed on the basis of the stimuli-response relationships ([3][4] for example).  However, speculating the functional role of a model neuronal network under natural conditions is not straightforward because images received by the retina under natural conditions are far more complex spatiotemporally than those used by physiological experiments. One of the feasible ways to speculate the functional role of model networks is simulating the activities of the model networks responding to natural images in real time. Although computer simulations using a sequential digital computing system, such as a personal computer (PC), have been used to examine the behavior of model neuronal networks [5], the high computational cost of simulating neuronal networks in which neurons have a bunch of connections does not allow simulators to reproduce neural activities in real time. Computer architecture itself has to be improved to run a real-time simulation. Employing parallel analog circuits, such as bio-inspired silicon retinas [6][7][8][9] or silicon early visual systems [10], is a feasible solution to run a real-time simulation because analog circuits offer instantaneous predetermined spatio-temporal processing required to simulate some parts of the retinal and cortical circuits. However, their built-in analog circuits lack configurability, which is also important for neural simulation. Configurability needs to be compensated by programmable systems. To fulfill the requirements of the visual system simulator described above, retina emulators with analog-digital mixed architecture has been proposed [11][12][13]. These emulators employed a silicon retina and a PC [11][12] or a field-programmable gate array (FPGA) [13] to take advantages of both analog and digital systems. In the present study, we further developed the emulator to include simulation of a part of cortical circuits. To reproduce the neural activities of both the retinal and cortical circuits, we combined the retina emulator [13] and the SpiNNaker chips [14], in which multiple CPUs simulate spiking neural networks in parallel and in real time. This mixed analog-digital architecture with multiple parallel processing techniques is suitable for simulating visual systems.  2 VISUAL SYSTEM EMULATOR 2.1 Hardware architecture Fig. 1 shows the diagram of the structure of the visual system emulator developed in this study. The emulator is composed of a retina emulator [13], a SpiNNaker interface (I/F) board, and a SpiNNaker [14] board. The appearance of the emulator is shown in Fig. 2. This emulator reproduces activities of both neuronal networks operating with graded potentials and those operating with spikes in real time, taking advantages of its mixed analog-digital architecture with multiple parallel processing techniques. The retina emulator [13] is composed of a silicon retina (Neuralimage Co., Ltd), an FPGA (Xilinx XC6SLX45), and signal converters. The silicon retina used here is an analog VLSI image sensor with image processing circuits. The design of a pixel is essentially the same as that described in [8]. It has 128 x 128 pixels, each of which is connected to four neighboring pixels by way of two layers of resistive networks, which apply a spatial low-pass filter to the whole pixel in parallel. This emulator generates simulated retinal spikes from images received through lens at 200 Hz. The generated spikes are sent to the SpiNNaker board by way of the SpiNNaker I/F board. The SpiNNaker board that has 4 SpiNNaker chips generates simulated cortical spikes from retinal spikes received from the retina emulator. Because each SpiNNaker chip contains 16 ARM cores for computation, 64 ARM cores simulate large scale neuronal networks in the visual cortex in parallel and in real time. The rich programmability of the SpiNNaker offers flexible network topology and configurable temporal properties of neurons.  2.2 Signal process flow The visual system emulator simulates the essence of computations carried out in the neuronal circuits in the retina and the primary visual cortex. Although many types of neuronal networks are programmable in SpiNNaker chips, we programmed only even-type simple cells to verify the operation of this simulation platform in this study. Fig. 3 depicts the process flow of the retina emulator. The spatial smoothing function mediated by the gap junctions of two types of retinal neurons (photoreceptors and horizontal cells) are simulated by two layers of the resistive networks in the silicon retina. The role of temporal filter TF1 is to simulate the delay of the lateral inhibition. TF1 is implemented with a first order infinite-impulse-response (IIR) filter. Temporal filter TF2 reflects temporal properties of neurons in the outer retina (photoreceptors and bipolar cells). TF2 is implemented with a linear summation of four first order IIR filters. The output signal of TF2 represents the responses of on-center bipolar cells. Spikes of ganglion cells are generated from the simulated response of bipolar cells based on a simple leaky-integrate-and-fire (LIF) model. Although this retina emulator can simulate multiple types of ganglion cells, we used only a single type of ganglion cell, which is called on-center sustained type, in this study.                              Fig. 1. Diagram of the system structure. The emulation system consists of a silicon retina, an FPGA, signal converters, and SpiNNaker chips. A PC is used for receiving the output data and for configuring the parameters.               Fig. 2. Appearance of the emulation system. The emulator consists of a retina emulator, a SpiNNaker I/F board, and a SpiNNaker board.   Each simulated simple cells [4] programmed in the SpiNNaker receives spikes from seven consecutive ganglion cells arranged horizontally or vertically, and   generates spikes based on LIF model. The aspect ratio of the simple cell in this configuration is about four; this value is comparable to the median of the aspect ratio of monkey simple cells [15].                    Fig. 3. Process flow diagram of the visual system emulator. Images are received by the photoreceptor in the silicon retina. Two layers of resistive networks simulate gap junction networks of two types of retinal neurons. Blocks TF1 and TF2 represent temporal filters simulating temporal properties of retinal neurons. Blocks LIF represent leaky-integrate-and-fire processing.  3 EXPERIMENTS AND RESULTS The neural activities simulated by the emulator were examined by presenting images on a liquid crystal display (LCD) and natural scenes.     Fig. 4(a) shows an input image used to examine the orientation selectivity of the simulated simple cells. Fig. 4 (b), (c), and (d) show simulated responses to this input image. Fig. 4(b) shows a spike pattern of simulated ganglion cells. Cells along the white lines generated spikes intensively. Fig. 4(c) and (d) show spike patterns of simulated simple cells that respond selectively to vertical and horizontal edges, respectively. Cells along the vertical line (c) or horizontal line (d) generated spikes intensively. Fig. 5(b),(c), and (d) show simulated responses to an input scene as shown in Fig. 5(a). Fig. 5(b) shows a spike                      (a)                   (b)                     (c)                   (d) Fig. 4. Input image and simulated spikes of the experiment to examine the orientation selectivity of the simulated simple cells. (a) Input image. (b) Simulated spikes generated by ganglion cells. (c) Simulated spikes generated by simple cells that respond selectively to vertical edges. (d) Simulated spikes generated by simple cells that respond selectively to horizontal edges.                     (a)                   (b)                    (c)                   (d) Fig. 5. Input scene and simulated spikes. (a) Input image. (b) Simulated spikes generated by ganglion cells.       (c) Simulated spikes generated by simple cells that respond selectively to vertical edges. (d) Simulated spikes generated by simple cells that respond selectively to horizontal edges. pattern of simulated ganglion cells. Cells around the contours of the display and the hand generated spikes intensively. Fig. 4(c) and (d) show spike patterns of simulated simple cells that respond selectively to vertical and horizontal edges, respectively. Cells around  vertical contours (c) or horizontal contours (d) generated spikes intensively.  4 CONCLUSION We developed a visual system emulator by using an analog-digital mixed architecture with multiple parallel processing techniques. This emulator, which simulates a part of the retinal and cortical circuits in real time and has rich configurability, could be a useful tool for speculating the functional roll of a particular neuronal circuit under natural conditions.  REFERENCES [1] Kuffler SW, (1953), Discharge Patterns and Functional Organization of Mammalian Retina, Journal of Neurophysiology 16: 37-68. [2] Hubel DH, Wiesel TN (1959), Receptive fields of single neurones in the cat's striate cortex, The Journal of Physiology 148: 574-591. [3] Enroth-Cugell C, Robson JG (1966), The contrast sensitivity of retinal ganglion cells of the cat, The Journal of Physiology 187: 517-552. [4] Hubel DH, Wiesel TN (1960), Receptive fields of optic nerve fibres in the spider monkey, The Journal of Physiology 154: 572-580. [5] Hines M (1984), Efficient computation of branched nerve equations, International journal of bio-medical computing 15: 69-76. [6] Mead CA, Mahowald M (1988), A silicon model of early visual processing, Neural Networks 1: 91-97. [7] Wu CY, Chiu CF (1995), A new structure of the 2-d silicon retina, IEEE Journal of Solid-State Circuits 30: 890-897.  [8] Kameda S, Yagi T (2003), An analog VLSI chip emulating sustained and transient response channels of the vertebrate retina, IEEE Transactions on Neural Networks 14: 1405-1412.  [9] Zaghloul K, Boahen K (2004), Optic nerve signals in a neuromorphic chip I: outer and inner retina models, IEEE Transactions on Biomedical Engineering 51: 657-666. [10] Delbruck T, Liu SC (2004) A silicon early visual system as a model animal, Vision Research 44: 2083-2089. [11] Hasegawa J, Yagi T (2008), Real-time emulation of neural images in the outer retinal circuit, The Journal of Physiological Sciences 58: 507-514. [12] Hasegawa J, Yagi T (2009), Real-time emulation of dynamic features of sustained and transient channels in vertebrate retina, in: Proc. of IEEE Biomedical Circuits and Systems Conference, Beijing, China, pp.197-200. [13] Okuno H, Hasegawa J, Yagi T (2013), A real-time emulation system for reconstructing spike activities of the retinal ganglion cells," in: Abstracts of Neuro 2013,  Kyoto, Japan. [14] Furber SB, Lester DV, Plana LA, et al. (2013), Overview of the SpiNNaker System Architecture, IEEE Transactions on Computers 62: 2454-2467. [15] Parker AJ, Hawken MJ (1988), Two-dimensional spatial structure of receptive fields in monkey striate cortex, Journal of the Optical Society of America A 5: 598-605. 

A real-time simulator of a biological visual system composed of a silicon retina and SpiNNaker chips

https://pure.manchester.ac.uk/ws/files/32297909/FULL_TEXT.PDF

A real-time simulator of a biological visual system composed of a silicon retina and SpiNNaker chips

Abstract

Similar works

Full text

Available Versions

The University of Manchester - Institutional Repository