1,085 research outputs found
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Bio-Inspired Stereo Vision Calibration for Dynamic Vision Sensors
Many advances have been made in the eld of computer vision. Several recent research trends
have focused on mimicking human vision by using a stereo vision system. In multi-camera systems, a
calibration process is usually implemented to improve the results accuracy. However, these systems generate
a large amount of data to be processed; therefore, a powerful computer is required and, in many cases,
this cannot be done in real time. Neuromorphic Engineering attempts to create bio-inspired systems that
mimic the information processing that takes place in the human brain. This information is encoded using
pulses (or spikes) and the generated systems are much simpler (in computational operations and resources),
which allows them to perform similar tasks with much lower power consumption, thus these processes
can be developed over specialized hardware with real-time processing. In this work, a bio-inspired stereovision
system is presented, where a calibration mechanism for this system is implemented and evaluated
using several tests. The result is a novel calibration technique for a neuromorphic stereo vision system,
implemented over specialized hardware (FPGA - Field-Programmable Gate Array), which allows obtaining
reduced latencies on hardware implementation for stand-alone systems, and working in real time.Ministerio de Economía y Competitividad TEC2016-77785-PMinisterio de Economía y Competitividad TIN2016-80644-
Spike-based VITE control with Dynamic Vision Sensor applied to an Arm Robot.
Spike-based motor control is very important in the
field of robotics and also for the neuromorphic engineering
community to bridge the gap between sensing / processing
devices and motor control without losing the spike philosophy
that enhances speed response and reduces power consumption.
This paper shows an accurate neuro-inspired spike-based system
composed of a DVS retina, a visual processing system that detects
and tracks objects, and a SVITE motor control, where everything
follows the spike-based philosophy. The control system is a spike
version of the neuroinspired open loop VITE control algorithm
implemented in a couple of FPGA boards: the first one runs the
algorithm and the second one drives the motors with spikes. The
robotic platform is a low cost arm with four degrees of freedom.Ministerio de Ciencia e Innovación TEC2009-10639-C04-02/01Ministerio de Economía y Competitividad TEC2012-37868-C04-02/0
An AER handshake-less modular infrastructure PCB with x8 2.5Gbps LVDS serial links
Nowadays spike-based brain processing emulation is
taking off. Several EU and others worldwide projects are
demonstrating this, like SpiNNaker, BrainScaleS, FACETS, or
NeuroGrid. The larger the brain process emulation on silicon is,
the higher the communication performance of the hosting
platforms has to be. Many times the bottleneck of these system
implementations is not on the performance inside a chip or a
board, but in the communication between boards. This paper
describes a novel modular Address-Event-Representation (AER)
FPGA-based (Spartan6) infrastructure PCB (the AER-Node
board) with 2.5Gbps LVDS high speed serial links over SATA
cables that offers a peak performance of 32-bit 62.5Meps (Mega
events per second) on board-to-board communications. The
board allows back compatibility with parallel AER devices
supporting up to x2 28-bit parallel data with asynchronous
handshake. These boards also allow modular expansion
functionality through several daughter boards. The paper is
focused on describing in detail the LVDS serial interface and
presenting its performance.Ministerio de Ciencia e Innovación TEC2009-10639-C04-02/01Ministerio de Economía y Competitividad TEC2012-37868-C04-02/01Junta de Andalucía TIC-6091Ministerio de Economía y Competitividad PRI-PIMCHI-2011-076
DART: Distribution Aware Retinal Transform for Event-based Cameras
We introduce a generic visual descriptor, termed as distribution aware
retinal transform (DART), that encodes the structural context using log-polar
grids for event cameras. The DART descriptor is applied to four different
problems, namely object classification, tracking, detection and feature
matching: (1) The DART features are directly employed as local descriptors in a
bag-of-features classification framework and testing is carried out on four
standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS,
NCaltech-101). (2) Extending the classification system, tracking is
demonstrated using two key novelties: (i) For overcoming the low-sample problem
for the one-shot learning of a binary classifier, statistical bootstrapping is
leveraged with online learning; (ii) To achieve tracker robustness, the scale
and rotation equivariance property of the DART descriptors is exploited for the
one-shot learning. (3) To solve the long-term object tracking problem, an
object detector is designed using the principle of cluster majority voting. The
detection scheme is then combined with the tracker to result in a high
intersection-over-union score with augmented ground truth annotations on the
publicly available event camera dataset. (4) Finally, the event context encoded
by DART greatly simplifies the feature correspondence problem, especially for
spatio-temporal slices far apart in time, which has not been explicitly tackled
in the event-based vision domain.Comment: 12 pages, revision submitted to TPAMI in Nov 201
Retinal ganglion cell software and FPGA model implementation for object detection and tracking
This paper describes the software and FPGA
implementation of a Retinal Ganglion Cell model which detects
moving objects. It is shown how this processing, in conjunction
with a Dynamic Vision Sensor as its input, can be used to
extrapolate information about object position. Software-wise, a
system based on an array of these of RGCs has been developed in
order to obtain up to two trackers. These can track objects in a
scene, from a still observer, and get inhibited when saccadic
camera motion happens. The entire processing takes on average
1000 ns/event. A simplified version of this mechanism, with a mean
latency of 330 ns/event, at 50 MHz, has also been implemented in
a Spartan6 FPGA.European Commission FP7-ICT-600954Ministerio de Economía y Competitividad TEC2012-37868-C04-02Junta de Andalucía P12-TIC-130
- …