30 research outputs found
A USB3.0 FPGA Event-based Filtering and Tracking Framework for Dynamic Vision Sensors
Dynamic vision sensors (DVS) are frame-free sensors
with an asynchronous variable-rate output that is ideal for hard
real-time dynamic vision applications under power and latency
constraints. Post-processing of the digital sensor output can
reduce sensor noise, extract low level features, and track objects
using simple algorithms that have previously been implemented
in software. In this paper we present an FPGA-based framework
for event-based processing that allows uncorrelated-event noise
removal and real-time tracking of multiple objects, with dynamic
capabilities to adapt itself to fast or slow and large or small
objects. This framework uses a new hardware platform based on
a Lattice FPGA which filters the sensor output and which then
transmits the results through a super-speed Cypress FX3 USB
microcontroller interface to a host computer. The packets of
events and timestamps are transmitted to the host computer at
rates of 10 Mega events per second. Experimental results are
presented that demonstrate a low latency of 10us for tracking
and computing the center of mass of a detected object.Ministerio de Economía y Competitividad TEC2012-37868-C04-0
Real-time motor rotation frequency detection with event-based visual and spike-based auditory AER sensory integration for FPGA
Multisensory integration is commonly
used in various robotic areas to collect more
environmental information using different and
complementary types of sensors. Neuromorphic
engineers mimics biological systems behavior to
improve systems performance in solving engineering
problems with low power consumption. This work
presents a neuromorphic sensory integration scenario
for measuring the rotation frequency of a motor using
an AER DVS128 retina chip (Dynamic Vision Sensor)
and a stereo auditory system on a FPGA completely
event-based. Both of them transmit information with
Address-Event-Representation (AER). This
integration system uses a new AER monitor hardware
interface, based on a Spartan-6 FPGA that allows two
operational modes: real-time (up to 5 Mevps through
USB2.0) and data logger mode (up to 20Mevps for
33.5Mev stored in onboard DDR RAM). The sensory
integration allows reducing prediction error of the
rotation speed of the motor since audio processing
offers a concrete range of rpm, while DVS can be
much more accurate.Ministerio de Economía y Competitividad TEC2012-37868-C04-02/0
Neuromorphic Approach Sensitivity Cell Modeling and FPGA Implementation
Neuromorphic engineering takes inspiration from biology to
solve engineering problems using the organizing principles of biological
neural computation. This field has demonstrated success in sensor based
applications (vision and audition) as well in cognition and actuators.
This paper is focused on mimicking an interesting functionality of the
retina that is computed by one type of Retinal Ganglion Cell (RGC).
It is the early detection of approaching (expanding) dark objects. This
paper presents the software and hardware logic FPGA implementation
of this approach sensitivity cell. It can be used in later cognition layers as
an attention mechanism. The input of this hardware modeled cell comes
from an asynchronous spiking Dynamic Vision Sensor, which leads to an
end-to-end event based processing system. The software model has been
developed in Java, and computed with an average processing time per
event of 370 ns on a NUC embedded computer. The output firing rate
for an approaching object depends on the cell parameters that represent
the needed number of input events to reach the firing threshold. For the
hardware implementation on a Spartan6 FPGA, the processing time is
reduced to 160 ns/event with the clock running at 50 MHz.Ministerio de Economía y Competitividad TEC2016-77785-PUnión Europea FP7-ICT-60095
Retinal ganglion cell software and FPGA model implementation for object detection and tracking
This paper describes the software and FPGA
implementation of a Retinal Ganglion Cell model which detects
moving objects. It is shown how this processing, in conjunction
with a Dynamic Vision Sensor as its input, can be used to
extrapolate information about object position. Software-wise, a
system based on an array of these of RGCs has been developed in
order to obtain up to two trackers. These can track objects in a
scene, from a still observer, and get inhibited when saccadic
camera motion happens. The entire processing takes on average
1000 ns/event. A simplified version of this mechanism, with a mean
latency of 330 ns/event, at 50 MHz, has also been implemented in
a Spartan6 FPGA.European Commission FP7-ICT-600954Ministerio de Economía y Competitividad TEC2012-37868-C04-02Junta de Andalucía P12-TIC-130
Low Latency Event-Based Filtering and Feature Extraction for Dynamic Vision Sensors in Real-Time FPGA Applications
Dynamic Vision Sensor (DVS) pixels produce an asynchronous variable-rate address-event
output that represents brightness changes at the pixel. Since these sensors produce frame-free output, they
are ideal for real-time dynamic vision applications with real-time latency and power system constraints.
Event-based ltering algorithms have been proposed to post-process the asynchronous event output to
reduce sensor noise, extract low level features, and track objects, among others. These postprocessing
algorithms help to increase the performance and accuracy of further processing for tasks such as classi cation
using spike-based learning (ie. ConvNets), stereo vision, and visually-servoed robots, etc. This paper
presents an FPGA-based library of these postprocessing event-based algorithms with implementation details;
speci cally background activity (noise) ltering, pixel masking, object motion detection and object tracking.
The latencies of these lters on the Field Programmable Gate Array (FPGA) platform are below 300ns with
an average latency reduction of 188% (maximum of 570%) over the software versions running on a desktop
PC CPU. This open-source event-based lter IP library for FPGA has been tested on two different platforms
and scenarios using different synthesis and implementation tools for Lattice and Xilinx vendors
ED-Scorbot: A Robotic test-bed Framework for FPGA-based Neuromorphic systems
Neuromorphic engineering is a growing and
promising discipline nowadays. Neuro-inspiration and
brain understanding applied to solve engineering
problems is boosting new architectures, solutions and
products today. The biological brain and neural systems
process information at relatively low speeds through
small components, called neurons, and it is impressive how
they connect each other to construct complex
architectures to solve in a quasi-instantaneous way
visual and audio processing tasks, object detection and
tracking, target approximation, grasping…, etc., with very
low power. Neuromorphs are beginning to be very promising
for a new era in the development of new sensors,
processors, robots and software systems that mimic
these biological systems. The event-driven Scorbot (EDScorbot)
is a robotic arm plus a set of FPGA / microcontroller’s
boards and a library of FPGA logic joined in a completely
event-based framework (spike-based) from the sensors to the
actuators. It is located in Seville (University of Seville) and
can be used remotely. Spike-based commands, through
neuro-inspired motor controllers, can be sent to the
robot after visual processing object detection and
tracking for grasping or manipulation, after complex
visual and audio-visual sensory fusion, or after performing
a learning task. Thanks to the cascade FPGA
architecture through the Address-Event-Representation
(AER) bus, supported by specialized boards, resources for
algorithms implementation are not limited.Ministerio de Economía y Competitividad TEC2012-37868-C04-02Junta de Andalucía P12-TIC-130
Dynamic Vision Sensor integration on FPGA-based CNN accelerators for high-speed visual classification
Deep-learning is a cutting edge theory that is being applied to many fields.
For vision applications the Convolutional Neural Networks (CNN) are demanding
significant accuracy for classification tasks. Numerous hardware accelerators
have populated during the last years to improve CPU or GPU based solutions.
This technology is commonly prototyped and tested over FPGAs before being
considered for ASIC fabrication for mass production. The use of commercial
typical cameras (30fps) limits the capabilities of these systems for high speed
applications. The use of dynamic vision sensors (DVS) that emulate the behavior
of a biological retina is taking an incremental importance to improve this
applications due to its nature, where the information is represented by a
continuous stream of spikes and the frames to be processed by the CNN are
constructed collecting a fixed number of these spikes (called events). The
faster an object is, the more events are produced by DVS, so the higher is the
equivalent frame rate. Therefore, these DVS utilization allows to compute a
frame at the maximum speed a CNN accelerator can offer. In this paper we
present a VHDL/HLS description of a pipelined design for FPGA able to collect
events from an Address-Event-Representation (AER) DVS retina to obtain a
normalized histogram to be used by a particular CNN accelerator, called
NullHop. VHDL is used to describe the circuit, and HLS for computation blocks,
which are used to perform the normalization of a frame needed for the CNN.
Results outperform previous implementations of frames collection and
normalization using ARM processors running at 800MHz on a Zynq7100 in both
latency and power consumption. A measured 67% speedup factor is presented for a
Roshambo CNN real-time experiment running at 160fps peak rate.Comment: 7 page
NAVIS: Neuromorphic Auditory VISualizer Tool
This software presents diverse utilities to perform the first post-processing layer taking the neuromorphic auditory sensors (NAS) information. The used NAS implements in FPGA a cascade filters architecture, imitating the behavior of the basilar membrane and inner hair cells and working with the sound information decomposed into its frequency components as spike streams. The well-known neuromorphic hardware interface Address-Event-Representation (AER) is used to propagate auditory information out of the NAS, emulating the auditory vestibular nerve. Using the information packetized into aedat files, which are generated through the jAER software plus an AER to USB computer interface, NAVIS implements a set of graphs that allows to represent the auditory information as cochleograms, histograms, sonograms, etc. It can also split the auditory information into different sets depending on the activity level of the spike streams. The main contribution of this software tool is that it allows complex audio post-processing treatments and representations, which is a novelty for spike-based systems in the neuromorphic community and it will help neuromorphic engineers to build sets for training spiking neural networks (SNN).Ministerio de Economía y Competitividad TEC2012-37868-C04-0
Bio-Inspired Stereo Vision Calibration for Dynamic Vision Sensors
Many advances have been made in the eld of computer vision. Several recent research trends
have focused on mimicking human vision by using a stereo vision system. In multi-camera systems, a
calibration process is usually implemented to improve the results accuracy. However, these systems generate
a large amount of data to be processed; therefore, a powerful computer is required and, in many cases,
this cannot be done in real time. Neuromorphic Engineering attempts to create bio-inspired systems that
mimic the information processing that takes place in the human brain. This information is encoded using
pulses (or spikes) and the generated systems are much simpler (in computational operations and resources),
which allows them to perform similar tasks with much lower power consumption, thus these processes
can be developed over specialized hardware with real-time processing. In this work, a bio-inspired stereovision
system is presented, where a calibration mechanism for this system is implemented and evaluated
using several tests. The result is a novel calibration technique for a neuromorphic stereo vision system,
implemented over specialized hardware (FPGA - Field-Programmable Gate Array), which allows obtaining
reduced latencies on hardware implementation for stand-alone systems, and working in real time.Ministerio de Economía y Competitividad TEC2016-77785-PMinisterio de Economía y Competitividad TIN2016-80644-