Point-to-point connectivity between neuromorphic chips using address events

This paper discusses connectivity between neuromorphic chips, which use the timing of fixed-height fixed-width pulses to encode information. Address-events (log2 (N)-bit packets that uniquely identify one of N neurons) are used to transmit these pulses in real time on a random-access time-multiplexed communication channel. Activity is assumed to consist of neuronal ensembles--spikes clustered in space and in time. This paper quantifies tradeoffs faced in allocating bandwidth, granting access, and queuing, as well as throughput requirements, and concludes that an arbitered channel design is the best choice.The arbitered channel is implemented with a formal design methodology for asynchronous digital VLSI CMOS systems, after introducing the reader to this top-down synthesis technique. Following the evolution of three generations of designs, it is shown how the overhead of arbitrating, and encoding and decoding, can be reduced in area (from N to √N) by organizing neurons into rows and columns, and reduced in time (from log2 (N) to 2) by exploiting locality in the arbiter tree and in the row–column architecture, and clustered activity. Throughput is boosted by pipelining and by reading spikes in parallel. Simple techniques that reduce crosstalk in these mixed analog–digital systems are described

Bio-Inspired Computer Vision: Towards a Synergistic Approach of Artificial and Biological Vision

To appear in CVIUStudies in biological vision have always been a great source of inspiration for design of computer vision algorithms. In the past, several successful methods were designed with varying degrees of correspondence with biological vision studies, ranging from purely functional inspiration to methods that utilise models that were primarily developed for explaining biological observations. Even though it seems well recognised that computational models of biological vision can help in design of computer vision algorithms, it is a non-trivial exercise for a computer vision researcher to mine relevant information from biological vision literature as very few studies in biology are organised at a task level. In this paper we aim to bridge this gap by providing a computer vision task centric presentation of models primarily originating in biological vision studies. Not only do we revisit some of the main features of biological vision and discuss the foundations of existing computational studies modelling biological vision, but also we consider three classical computer vision tasks from a biological perspective: image sensing, segmentation and optical flow. Using this task-centric approach, we discuss well-known biological functional principles and compare them with approaches taken by computer vision. Based on this comparative analysis of computer and biological vision, we present some recent models in biological vision and highlight a few models that we think are promising for future investigations in computer vision. To this extent, this paper provides new insights and a starting point for investigators interested in the design of biology-based computer vision algorithms and pave a way for much needed interaction between the two communities leading to the development of synergistic models of artificial and biological vision

A neurobiological and computational analysis of target discrimination in visual clutter by the insect visual system.

Some insects have the capability to detect and track small moving objects, often against cluttered moving backgrounds. Determining how this task is performed is an intriguing challenge, both from a physiological and computational perspective. Previous research has characterized higher-order neurons within the fly brain known as 'small target motion detectors‘ (STMD) that respond selectively to targets, even within complex moving surrounds. Interestingly, these cells still respond robustly when the velocity of the target is matched to the velocity of the background (i.e. with no relative motion cues). We performed intracellular recordings from intermediate-order neurons in the fly visual system (the medulla). These full-wave rectifying, transient cells (RTC) reveal independent adaptation to luminance changes of opposite signs (suggesting separate 'on‘ and 'off‘ channels) and fast adaptive temporal mechanisms (as seen in some previously described cell types). We show, via electrophysiological experiments, that the RTC is temporally responsive to rapidly changing stimuli and is well suited to serving an important function in a proposed target-detecting pathway. To model this target discrimination, we use high dynamic range (HDR) natural images to represent 'real-world‘ luminance values that serve as inputs to a biomimetic representation of photoreceptor processing. Adaptive spatiotemporal high-pass filtering (1st-order interneurons) shapes the transient 'edge-like‘ responses, useful for feature discrimination. Following this, a model for the RTC implements a nonlinear facilitation between the rapidly adapting, and independent polarity contrast channels, each with centre-surround antagonism. The recombination of the channels results in increased discrimination of small targets, of approximately the size of a single pixel, without the need for relative motion cues. This method of feature discrimination contrasts with traditional target and background motion-field computations. We show that our RTC-based target detection model is well matched to properties described for the higher-order STMD neurons, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear 'matched filter‘ to successfully detect many targets from the background. The model produces robust target discrimination across a biologically plausible range of target sizes and a range of velocities. We show that the model for small target motion detection is highly correlated to the velocity of the stimulus but not other background statistics, such as local brightness or local contrast, which normally influence target detection tasks. From an engineering perspective, we examine model elaborations for improved target discrimination via inhibitory interactions from correlation-type motion detectors, using a form of antagonism between our feature correlator and the more typical motion correlator. We also observe that a changing optimal threshold is highly correlated to the value of observer ego-motion. We present an elaborated target detection model that allows for implementation of a static optimal threshold, by scaling the target discrimination mechanism with a model-derived velocity estimation of ego-motion. Finally, we investigate the physiological relevance of this target discrimination model. We show that via very subtle image manipulation of the visual stimulus, our model accurately predicts dramatic changes in observed electrophysiological responses from STMD neurons.Thesis (Ph.D.) - University of Adelaide, School of Molecular and Biomedical Science, 200

HSMD: an object motion detection algorithm using a Hybrid Spiking Neural Network Architecture

The detection of moving objects is a trivial task performed by vertebrate retinas, yet a complex computer vision task. Object-motion-sensitive ganglion cells (OMS-GC) are specialised cells in the retina that sense moving objects. OMS-GC take as input continuous signals and produce spike patterns as output, that are transmitted to the Visual Cortex via the optic nerve. The Hybrid Sensitive Motion Detector (HSMD) algorithm proposed in this work enhances the GSOC dynamic background subtraction (DBS) algorithm with a customised 3-layer spiking neural network (SNN) that outputs spiking responses akin to the OMS-GC. The algorithm was compared against existing background subtraction (BS) approaches, available on the OpenCV library, specifically on the 2012 change detection (CDnet2012) and the 2014 change detection (CDnet2014) benchmark datasets. The results show that the HSMD was ranked overall first among the competing approaches and has performed better than all the other algorithms on four of the categories across all the eight test metrics. Furthermore, the HSMD proposed in this paper is the first to use an SNN to enhance an existing state of the art DBS (GSOC) algorithm and the results demonstrate that the SNN provides near real-time performance in realistic applications

Towards Computational Models and Applications of Insect Visual Systems for Motion Perception: A Review

Motion perception is a critical capability determining a variety of aspects of insects' life, including avoiding predators, foraging and so forth. A good number of motion detectors have been identified in the insects' visual pathways. Computational modelling of these motion detectors has not only been providing effective solutions to artificial intelligence, but also benefiting the understanding of complicated biological visual systems. These biological mechanisms through millions of years of evolutionary development will have formed solid modules for constructing dynamic vision systems for future intelligent machines. This article reviews the computational motion perception models originating from biological research of insects' visual systems in the literature. These motion perception models or neural networks comprise the looming sensitive neuronal models of lobula giant movement detectors (LGMDs) in locusts, the translation sensitive neural systems of direction selective neurons (DSNs) in fruit flies, bees and locusts, as well as the small target motion detectors (STMDs) in dragonflies and hover flies. We also review the applications of these models to robots and vehicles. Through these modelling studies, we summarise the methodologies that generate different direction and size selectivity in motion perception. At last, we discuss about multiple systems integration and hardware realisation of these bio-inspired motion perception models

Matrix Transform Imager Architecture for On-Chip Low-Power Image Processing

Camera-on-a-chip systems have tried to include carefully chosen signal processing units for better functionality, performance and also to broaden the applications they can be used for. Image processing sensors have been possible due advances in CMOS active pixel sensors (APS) and neuromorphic focal plane imagers. Some of the advantages of these systems are compact size, high speed and parallelism, low power dissipation, and dense system integration. One can envision using these chips for portable and inexpensive video cameras on hand-held devices like personal digital assistants (PDA) or cell-phones In neuromorphic modeling of the retina it would be very nice to have processing capabilities at the focal plane while retaining the density of typical APS imager designs. Unfortunately, these two goals have been mostly incompatible. We introduce our MAtrix Transform Imager Architecture (MATIA) that uses analog floating--gate devices to make it possible to have computational imagers with high pixel densities. The core imager performs computations at the pixel plane, but still has a fill-factor of 46 percent - comparable to the high fill-factors of APS imagers. The processing is performed continuously on the image via programmable matrix operations that can operate on the entire image or blocks within the image. The resulting data-flow architecture can directly perform all kinds of block matrix image transforms. Since the imager operates in the subthreshold region and thus has low power consumption, this architecture can be used as a low-power front end for any system that utilizes these computations. Various compression algorithms (e.g. JPEG), that use block matrix transforms, can be implemented using this architecture. Since MATIA can be used for gradient computations, cheap image tracking devices can be implemented using this architecture. Other applications of this architecture can range from stand-alone universal transform imager systems to systems that can compute stereoscopic depth.Ph.D.Committee Chair: Hasler, Paul; Committee Member: David Anderson; Committee Member: DeWeerth, Steve; Committee Member: Jackson, Joel; Committee Member: Smith, Mar

Computational models of object motion detectors accelerated using FPGA technology

The detection of moving objects is a trivial task when performed by vertebrate retinas, yet a complex computer vision task. This PhD research programme has made three key contributions, namely: 1) a multi-hierarchical spiking neural network (MHSNN) architecture for detecting horizontal and vertical movements, 2) a Hybrid Sensitive Motion Detector (HSMD) algorithm for detecting object motion and 3) the Neuromorphic Hybrid Sensitive Motion Detector (NeuroHSMD) , a real-time neuromorphic implementation of the HSMD algorithm. The MHSNN is a customised 4 layers Spiking Neural Network (SNN) architecture designed to reflect the basic connectivity, similar to canonical behaviours found in the majority of vertebrate retinas (including human retinas). The architecture, was trained using images from a custom dataset generated in laboratory settings. Simulation results revealed that each cell model is sensitive to vertical and horizontal movements, with a detection error of 6.75% contrasted against the teaching signals (expected output signals) used to train the MHSNN. The experimental evaluation of the methodology shows that the MH SNN was not scalable because of the overall number of neurons and synapses which lead to the development of the HSMD. The HSMD algorithm enhanced an existing Dynamic Background subtraction (DBS) algorithm using a customised 3-layer SNN. The customised 3-layer SNN was used to stabilise the foreground information of moving objects in the scene, which improves the object motion detection. The algorithm was compared against existing background subtraction approaches, available on the Open Computer Vision (OpenCV) library, specifically on the 2012 Change Detection (CDnet2012) and the 2014 Change Detection (CDnet2014) benchmark datasets. The accuracy results show that the HSMD was ranked overall first and performed better than all the other benchmarked algorithms on four of the categories, across all eight test metrics. Furthermore, the HSMD is the first to use an SNN to enhance the existing dynamic background subtraction algorithm without a substantial degradation of the frame rate, being capable of processing images 720 × 480 at 13.82 Frames Per Second (fps) (CDnet2014) and 720 × 480 at 13.92 fps (CDnet2012) on a High Performance computer (96 cores and 756 GB of RAM). Although the HSMD analysis shows good Percentage of Correct Classifications (PCC) on the CDnet2012 and CDnet2014, it was identified that the 3-layer customised SNN was the bottleneck, in terms of speed, and could be improved using dedicated hardware. The NeuroHSMD is thus an adaptation of the HSMD algorithm whereby the SNN component has been fully implemented on dedicated hardware [Terasic DE10-pro Field-Programmable Gate Array (FPGA) board]. Open Computer Language (OpenCL) was used to simplify the FPGA design flow and allow the code portability to other devices such as FPGA and Graphical Processing Unit (GPU). The NeuroHSMD was also tested against the CDnet2012 and CDnet2014 datasets with an acceleration of 82% over the HSMD algorithm, being capable of processing 720 × 480 images at 28.06 fps (CDnet2012) and 28.71 fps (CDnet2014)

Neuromorphic models for biological photoreceptors

Biological visual processing is extremely flexible and provides pixelby- pixel adaptation. Millennia of evolution and natural selection have provided inspiration for robust, efficient and elegant solutions in artificial visual system designs. Physiological studies have shown that non-linear adaptation of biological visual processing is evident even at the first stage of the visual system pathway. Theory and modelling have shown that adaptation in the early visual processing is required to compress the high bandwidth visual environment into a sensible form prior to transmission via the limited bandwidth neuron channels. However, many current bio-inspired visual systems have neglected the importance of having a reliable early stage of visual processing. Having a robust and reliable early stage design not only provides a better mimic of the biology, but also allows better design and understanding of higher order neurons in the visual system pathway. (Chapter 3: A Non-linear Adaptive Artificial Photoreceptor Circuit - Design and Implementation) The primary aim of this work was to design and implement an elaborated artificial photoreceptor circuit which faithfully mimics the actual biological photoreceptors, using standard analogue discrete electronic components. I have incorporated several key features of the biological photoreceptors in the implementation, such as non-linear adaptation to background luminance, adaptive frequency response and logarithmic encoding of luminance. Initial parameters for the key features of the model were based on existing literature and fine tuning of the circuit was done after analysis of actual recordings from biological photoreceptors. (Chapter 2: Dimmable Voltage-Controlled High Current LED Driver System for Vision Science Experiments) The visual stimulus was a critical component in performing the vision experiments, and has historically been a limiting factor in performing experiments which ask critical questions about responses to complicated scenes, such as natural environments. The ability to reproduce the large dynamic range of the real-world luminance was important to correctly test the performance of the model. I evaluated the performance of several existing light emitting diode (LED) drivers and commercial products and found that none of them provided adequate dynamic range and freedom from noise. I therefore designed and implemented a stable multi-channel, high-current LED driver that allowed creation of light stimuli with inexpensive analogue discrete electronic components, and was used for the experiments described in this thesis. This LED driver, which was properly calibrated to the real-world luminance, was used in conjunction with a standard commercial data acquisition card. (An Elaborated Electronic Prototype of a Biological Photoreceptor - Steady-state Analysis (Chapter 4) & Dynamic Analysis (Chapter 5)) I performed electrophysiological experiments measuring the responses of the intact hoverfly photoreceptor cells (Rl-6) using both characterised and dynamic (naturalistic) stimuli. The analysed data were used to fine tune the circuit parameters in order to realise a faithful mimic of the actual biological photoreceptors. Similar experiments were performed on the artificial photoreceptor circuit to thoroughly evaluate the robustness and performance of the circuit against actual biological photoreceptors. Correlation and coherence analyses were used to measure the performance of the circuit with respect to its biological counterpart in both time and frequency domains respectively. Chapter 6: Early Visual Processing Maximises Information for Higher Order Neurons) The artificial photoreceptor circuit was then further evaluated against a complex natural movie scene in which the full dynamic range of the original scenario was maintained. Again, I performed experiments on both the circuit and actual biological photoreceptors. Correlation and coherence analyses of the circuit against the biological photoreceptors showed that the circuit was robust and reliable even under complex naturalistic conditions. I managed to design and implement an add-on electronic circuit to the elaborated photoreceptor circuit that crudely mimicked the temporal high-pass nature of the second order Large Monopolar Cell (LMC) in order to observe how the non-linear features in the early stage of visual processing assists higher order neurons in efficiently coding visual information. Based on this research, I found that the first stage of visual processing consists of numerous non-linearities, which have been proven to provide optimal coding of visual information. The variable frequency response curve of the hoverfly, Eristalis tenax was mapped out against large range of background luminance. Previous studies have suggested that such variability in frequency response was to improve signal transmission quality in the insect visual pathway, even though I have not made any quantitative measurements of the improvements. I also found that high dynamic range images (32-bit floating point numbers) are better representations of the real-world luminance for naturalistic visual experiments compared to the conventional 8-bit images. I have successfully implemented a circuit that faithfully mimicked the biological photoreceptors and it has been evaluated against characterised and dynamic stimuli. I found that my circuit design was far better than using just a normal linear phototransducer as the front-end of a vision system as it is more capable of compressing visual information in a way which maximises the information content before transmission to higher order neurons.Thesis (Ph.D.) -- University of Adelaide, School of Molecular and Biomedical Sciences, Discipline of Physiology, 2007

An Optoelectronic Stimulator for Retinal Prosthesis

Retinal prostheses require the presence of viable population of cells in the inner retina. Evaluations of retina with Age-Related Macular Degeneration (AMD) and Retinitis Pigmentosa (RP) have shown a large number of cells remain in the inner retina compared with the outer retina. Therefore, vision loss caused by AMD and RP is potentially treatable with retinal prostheses. Photostimulation based retinal prostheses have shown many advantages compared with retinal implants. In contrary to electrode based stimulation, light does not require mechanical contact. Therefore, the system can be completely external and not does have the power and degradation problems of implanted devices. In addition, the stimulating point is flexible and does not require a prior decision on the stimulation location. Furthermore, a beam of light can be projected on tissue with both temporal and spatial precision. This thesis aims at fi nding a feasible solution to such a system. Firstly, a prototype of an optoelectronic stimulator was proposed and implemented by using the Xilinx Virtex-4 FPGA evaluation board. The platform was used to demonstrate the possibility of photostimulation of the photosensitized neurons. Meanwhile, with the aim of developing a portable retinal prosthesis, a system on chip (SoC) architecture was proposed and a wide tuning range sinusoidal voltage-controlled oscillator (VCO) which is the pivotal component of the system was designed. The VCO is based on a new designed Complementary Metal Oxide Semiconductor (CMOS) Operational Transconductance Ampli er (OTA) which achieves a good linearity over a wide tuning range. Both the OTA and the VCO were fabricated in the AMS 0.35 µm CMOS process. Finally a 9X9 CMOS image sensor with spiking pixels was designed. Each pixel acts as an independent oscillator whose frequency is controlled by the incident light intensity. The sensor was fabricated in the AMS 0.35 µm CMOS Opto Process. Experimental validation and measured results are provided