601 research outputs found
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
An AER Spike-Processing Filter Simulator and Automatic VHDL Generator Based on Cellular Automata
Spike-based systems are neuro-inspired circuits implementations
traditionally used for sensory systems or sensor signal processing. Address-Event-
Representation (AER) is a neuromorphic communication protocol for transferring
asynchronous events between VLSI spike-based chips. These neuro-inspired
implementations allow developing complex, multilayer, multichip neuromorphic
systems and have been used to design sensor chips, such as retinas and cochlea,
processing chips, e.g. filters, and learning chips. Furthermore, Cellular Automata
(CA) is a bio-inspired processing model for problem solving. This approach
divides the processing synchronous cells which change their states at the same time
in order to get the solution. This paper presents a software simulator able to gather
several spike-based elements into the same workspace in order to test a CA
architecture based on AER before a hardware implementation. Furthermore this
simulator produces VHDL for testing the AER-CA into the FPGA of the USBAER
AER-tool.Ministerio de Ciencia e Innovación TEC2009-10639-C04-0
Visual Spike-based Convolution Processing with a Cellular Automata Architecture
this paper presents a first approach for
implementations which fuse the Address-Event-Representation
(AER) processing with the Cellular Automata using FPGA and
AER-tools. This new strategy applies spike-based convolution
filters inspired by Cellular Automata for AER vision
processing. Spike-based systems are neuro-inspired circuits
implementations traditionally used for sensory systems or
sensor signal processing. AER is a neuromorphic
communication protocol for transferring asynchronous events
between VLSI spike-based chips. These neuro-inspired
implementations allow developing complex, multilayer,
multichip neuromorphic systems and have been used to design
sensor chips, such as retinas and cochlea, processing chips, e.g.
filters, and learning chips. Furthermore, Cellular Automata is a
bio-inspired processing model for problem solving. This
approach divides the processing synchronous cells which
change their states at the same time in order to get the solution.Ministerio de Educación y Ciencia TEC2006-11730-C03-02Ministerio de Ciencia e Innovación TEC2009-10639-C04-02Junta de Andalucía P06-TIC-0141
Spike Processing on an Embedded Multi-task Computer: Image Reconstruction
There is an emerging philosophy, called Neuro-informatics, contained
in the Artificial Intelligence field, that aims to emulate how living beings do tasks
such as taking a decision based on the interpretation of an image by emulating spiking
neurons into VLSI designs and, therefore, trying to re-create the human brain at
its highest level. Address-Event-Representation (AER) is a communication protocol
that has embedded part of the processing. It is intended to transfer spikes between
bioinspired chips. An AER based system may consist of a hierarchical structure with
several chips that transmit spikes among them in real-time, while performing some
processing. There are several AER tools to help to develop and test AER based systems.
These tools require the use of a computer to allow the higher level processing of
the event information, reaching very high bandwidth at the AER communication level.
We propose the use of an embedded platform based on a multi-task operating system
to allow both, the AER communication and processing without the requirement of either
a laptop or a computer. In this paper, we present and study the performance of a
new philosophy of a frame-grabber AER tool based on a multi-task environment. This
embedded platform is based on the Intel XScale processor which is governed by an
embedded GNU/Linux system. We have connected and programmed it for processing
Address-Event information from a spiking generator.Ministerio de Educación y Ciencia TEC2006-11730-C03-0
DART: Distribution Aware Retinal Transform for Event-based Cameras
We introduce a generic visual descriptor, termed as distribution aware
retinal transform (DART), that encodes the structural context using log-polar
grids for event cameras. The DART descriptor is applied to four different
problems, namely object classification, tracking, detection and feature
matching: (1) The DART features are directly employed as local descriptors in a
bag-of-features classification framework and testing is carried out on four
standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS,
NCaltech-101). (2) Extending the classification system, tracking is
demonstrated using two key novelties: (i) For overcoming the low-sample problem
for the one-shot learning of a binary classifier, statistical bootstrapping is
leveraged with online learning; (ii) To achieve tracker robustness, the scale
and rotation equivariance property of the DART descriptors is exploited for the
one-shot learning. (3) To solve the long-term object tracking problem, an
object detector is designed using the principle of cluster majority voting. The
detection scheme is then combined with the tracker to result in a high
intersection-over-union score with augmented ground truth annotations on the
publicly available event camera dataset. (4) Finally, the event context encoded
by DART greatly simplifies the feature correspondence problem, especially for
spatio-temporal slices far apart in time, which has not been explicitly tackled
in the event-based vision domain.Comment: 12 pages, revision submitted to TPAMI in Nov 201
Recommended from our members
O(N)-Space Spatiotemporal Filter for Reducing Noise in Neuromorphic Vision Sensors
Neuromorphic vision sensors are an emerging technology inspired by how retina processing images. A neuromorphic vision sensor only reports when a pixel value changes rather than continuously outputting the value every frame as is done in an 'ordinary' Active Pixel Sensor (ASP). This move from a continuously sampled system to an asynchronous event driven one effectively allows for much faster sampling rates; it also fundamentally changes the sensor interface. In particular, these sensors are highly sensitive to noise, as any additional event reduces the bandwidth, and thus effectively lowers the sampling rate. In this work we introduce a novel spatiotemporal filter with O(N)O(N) memory complexity for reducing background activity noise in neuromorphic vision sensors. Our design consumes 10× less memory and has 100× reduction in error compared to previous designs. Our filter is also capable of recovering real events and can pass up to 180 percent more real events
From Vision Sensor to Actuators, Spike Based Robot Control through Address-Event-Representation
One field of the neuroscience is the neuroinformatic whose aim is to
develop auto-reconfigurable systems that mimic the human body and brain. In
this paper we present a neuro-inspired spike based mobile robot. From
commercial cheap vision sensors converted into spike information, through
spike filtering for object recognition, to spike based motor control models. A
two wheel mobile robot powered by DC motors can be autonomously
controlled to follow a line drown in the floor. This spike system has been
developed around the well-known Address-Event-Representation mechanism to
communicate the different neuro-inspired layers of the system. RTC lab has
developed all the components presented in this work, from the vision sensor, to
the robot platform and the FPGA based platforms for AER processing.Ministerio de Ciencia e Innovación TEC2006-11730-C03-02Junta de Andalucía P06-TIC-0141
An Event-Driven Multi-Kernel Convolution Processor Module for Event-Driven Vision Sensors
Event-Driven vision sensing is a new way of sensing
visual reality in a frame-free manner. This is, the vision sensor
(camera) is not capturing a sequence of still frames, as in conventional
video and computer vision systems. In Event-Driven sensors
each pixel autonomously and asynchronously decides when to
send its address out. This way, the sensor output is a continuous
stream of address events representing reality dynamically continuously
and without constraining to frames. In this paper we present
an Event-Driven Convolution Module for computing 2D convolutions
on such event streams. The Convolution Module has been
designed to assemble many of them for building modular and hierarchical
Convolutional Neural Networks for robust shape and
pose invariant object recognition. The Convolution Module has
multi-kernel capability. This is, it will select the convolution kernel
depending on the origin of the event. A proof-of-concept test prototype
has been fabricated in a 0.35 m CMOS process and extensive
experimental results are provided. The Convolution Processor has
also been combined with an Event-Driven Dynamic Vision Sensor
(DVS) for high-speed recognition examples. The chip can discriminate
propellers rotating at 2 k revolutions per second, detect symbols
on a 52 card deck when browsing all cards in 410 ms, or detect
and follow the center of a phosphor oscilloscope trace rotating at
5 KHz.Unión Europea 216777 (NABAB)Ministerio de Ciencia e Innovación TEC2009-10639-C04-0
- …