Poker-DVS and MNIST-DVS. Their History, How They Were Made, and Other Details

This article reports on two databases for event-driven object recognition using a Dynamic Vision Sensor (DVS). The first, which we call Poker-DVS and is being released together with this article, was obtained by browsing specially made poker card decks in front of a DVS camera for 2–4 s. Each card appeared on the screen for about 20–30 ms. The poker pips were tracked and isolated off-line to constitute the 131-recording Poker-DVS database. The second database, which we call MNIST-DVS and which was released in December 2013, consists of a set of 30,000 DVS camera recordings obtained by displaying 10,000 moving symbols from the standard MNIST 70,000-picture database on an LCD monitor for about 2–3 s each. Each of the 10,000 symbols was displayed at three different scales, so that event-driven object recognition algorithms could easily be tested for different object sizes. This article tells the story behind both databases, covering, among other aspects, details of how they work and the reasons for their creation. We provide not only the databases with corresponding scripts, but also the scripts and data used to generate the figures shown in this article (as Supplementary Material).Junta de Andalucía TIC-6091Ministerio de Economía y Competitividad TEC2012-37868-C04-01European Union FP7-604102, H2020-64409

A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems

We present a 32 32 pixels contrast retina microchip that provides its output as an address event representation (AER) stream. Spatial contrast is computed as the ratio between pixel photocurrent and a local average between neighboring pixels obtained with a diffuser network. This current-based computation produces an important amount of mismatch between neighboring pixels, because the currents can be as low as a few pico-amperes. Consequently, a compact calibration circuitry has been included to trimm each pixel. Measurements show a reduction in mismatch standard deviation from 57% to 6.6% (indoor light). The paper describes the design of the pixel with its spatial contrast computation and calibration sections. About one third of pixel area is used for a 5-bit calibration circuit. Area of pixel is 58 m 56 m, while its current consumption is about 20 nA at 1-kHz event rate. Extensive experimental results are provided for a prototype fabricated in a standard 0.35- m CMOS process.This work was supported by Spanish Research Grants TIC2003-08164-C03-01 (SAMANTA), TEC2006-11730-C03-01 (SAMANTA-II), and EU grant IST-2001-34124 (CAVIAR). JCS was supported by the I3P program of the Spanish Research Council. RSG was supported by a national grant from the Spanish Ministry of Education and Science.Peer reviewe

A low power CMOS imager based on time-to-first-spike encoding and fair AER

This paper presents a CMOS vision sensor based on a biologically inspired data representation referred to as Time-to-First-Spike (TFS) encoding combined with a fair Address Event Representation (AER) scheme. Our approach is different from conventional methods because the read-out of information is initiated by the pixel itself while access to the read-out bus is granted only once to each pixel after which it enters into a stand-by mode. This approach allows to greatly save dynamic power consumption and to extensively reduce inefficiencies due to periodical requests of the bus in the case of spiking pixels. Transmission bandwidth is thus significantly improved using the proposed circuitry. Each pixel includes only 14 transistors and occupies an area of 15 x 15 mu m(2), with a fill factor of 33% using 0.35 mu m process. The average current consumption is estimated to 10nA per pixel, which is 3 orders of magnitude lower compared with that of the spiking pixel

High speed event-based visual processing in the presence of noise

Standard machine vision approaches are challenged in applications where large amounts of noisy temporal data must be processed in real-time. This work aims to develop neuromorphic event-based processing systems for such challenging, high-noise environments. The novel event-based application-focused algorithms developed are primarily designed for implementation in digital neuromorphic hardware with a focus on noise robustness, ease of implementation, operationally useful ancillary signals and processing speed in embedded systems

Asynchroner CMOS–Bildsensor mit erweitertem Dynamikbereich und Unterdrückung zeitlich redundanter Daten: Asynchroner CMOS–Bildsensor mit erweitertem Dynamikbereich und Unterdrückung zeitlich redundanter Daten

Diese Arbeit befasst sich mit dem Entwurf eines asynchron arbeitenden, zeitbasierten CMOS–Bildsensors mit erhöhtem Dynamikbereich und Unterdrückung zeitlich redundanter Daten. Aufgrund immer kleinerer Strukturgrößen in modernen Prozessen zur Fertigung von Halbleitern und einer gleichzeitig physikalisch bedingt immer geringeren Skalierbarkeit konventioneller Bildsensoren wird es zunehmend möglich und praktikabel, Signalverarbeitungsansätze auf Pixelebene zu implementieren. Unter Berücksichtigung dieser Entwicklungen befasst sich die folgende Arbeit mit dem Entwurf eines neuartigen CMOS–Bildsensors mit nahezu vollständiger Unterdrückung zeitlich redundanter Daten auf Pixelebene. Jedes photosensitive Element in der Matrix arbeitet dabei vollkommen autonom. Es detektiert selbständig Änderungen in der Bestrahlung und gibt den Absolutwert nur beim Auftreten einer solchen Änderung mittels asynchroner Signalisierung nach außen. Darüber hinaus zeichnet sich der entwickelte Bildaufnehmer durch einen, gegenüber herkömmlichen Bildsensoren, deutlich erhöhten Dynamikbereich und eine niedrige Energieaufnahme aus, wodurch das Prinzip besonders für die Verwendung in Systemen für den mobilen Einsatz oder zur Durchführung von Überwachungsaufgaben geeignet ist. Die Realisierbarkeit des Konzepts wurde durch die erfolgreiche Implementierung eines entsprechenden Bildaufnehmers in einem Standard–CMOS–Prozess nachgewiesen. Durch die Größe des Designs von 304 x 240 Bildelementen, die den Umfang üblicher Prototypen-Realisierungen deutlich übersteigt, konnte speziell die Anwendbarkeit im Bereich größerer Sensorfelder gezeigt werden. Der Schaltkreis wurde erfolgreich getestet, wobei sowohl das Gesamtsystem als auch einzelne Schaltungsteile messtechnisch analysiert worden sind. Die nachgewiesene Bildqualität deckt sich dabei in guter Näherung mit den theoretischen Vorbetrachtungen

Bio-inspired electronics for micropower vision processing

Vision processing is a topic traditionally associated with neurobiology; known to encode, process and interpret visual data most effectively. For example, the human retina; an exquisite sheet of neurobiological wetware, is amongst the most powerful and efficient vision processors known to mankind. With improving integrated technologies, this has generated considerable research interest in the microelectronics community in a quest to develop effective, efficient and robust vision processing hardware with real-time capability. This thesis describes the design of a novel biologically-inspired hybrid analogue/digital vision chip ORASIS1 for centroiding, sizing and counting of enclosed objects. This chip is the first two-dimensional silicon retina capable of centroiding and sizing multiple objects2 in true parallel fashion. Based on a novel distributed architecture, this system achieves ultra-fast and ultra-low power operation in comparison to conventional techniques. Although specifically applied to centroid detection, the generalised architecture in fact presents a new biologically-inspired processing paradigm entitled: distributed asynchronous mixed-signal logic processing. This is applicable to vision and sensory processing applications in general that require processing of large numbers of parallel inputs, normally presenting a computational bottleneck. Apart from the distributed architecture, the specific centroiding algorithm and vision chip other original contributions include: an ultra-low power tunable edge-detection circuit, an adjustable threshold local/global smoothing network and an ON/OFF-adaptive spiking photoreceptor circuit. Finally, a concise yet comprehensive overview of photodiode design methodology is provided for standard CMOS technologies. This aims to form a basic reference from an engineering perspective, bridging together theory with measured results. Furthermore, an approximate photodiode expression is presented, aiming to provide vision chip designers with a basic tool for pre-fabrication calculations