188 research outputs found
Pseudo-labels for Supervised Learning on Dynamic Vision Sensor Data, Applied to Object Detection under Ego-motion
In recent years, dynamic vision sensors (DVS), also known as event-based
cameras or neuromorphic sensors, have seen increased use due to various
advantages over conventional frame-based cameras. Using principles inspired by
the retina, its high temporal resolution overcomes motion blurring, its high
dynamic range overcomes extreme illumination conditions and its low power
consumption makes it ideal for embedded systems on platforms such as drones and
self-driving cars. However, event-based data sets are scarce and labels are
even rarer for tasks such as object detection. We transferred discriminative
knowledge from a state-of-the-art frame-based convolutional neural network
(CNN) to the event-based modality via intermediate pseudo-labels, which are
used as targets for supervised learning. We show, for the first time,
event-based car detection under ego-motion in a real environment at 100 frames
per second with a test average precision of 40.3% relative to our annotated
ground truth. The event-based car detector handles motion blur and poor
illumination conditions despite not explicitly trained to do so, and even
complements frame-based CNN detectors, suggesting that it has learnt
generalized visual representations
Distributed Event-Based State Estimation for Networked Systems: An LMI-Approach
In this work, a dynamic system is controlled by multiple sensor-actuator
agents, each of them commanding and observing parts of the system's input and
output. The different agents sporadically exchange data with each other via a
common bus network according to local event-triggering protocols. From these
data, each agent estimates the complete dynamic state of the system and uses
its estimate for feedback control. We propose a synthesis procedure for
designing the agents' state estimators and the event triggering thresholds. The
resulting distributed and event-based control system is guaranteed to be stable
and to satisfy a predefined estimation performance criterion. The approach is
applied to the control of a vehicle platoon, where the method's trade-off
between performance and communication, and the scalability in the number of
agents is demonstrated.Comment: This is an extended version of an article to appear in the IEEE
Transactions on Automatic Control (additional parts in the Appendix
NeuroPod: a real-time neuromorphic spiking CPG applied to robotics
Initially, robots were developed with the aim of making our life easier, carrying
out repetitive or dangerous tasks for humans. Although they were able
to perform these tasks, the latest generation of robots are being designed
to take a step further, by performing more complex tasks that have been
carried out by smart animals or humans up to date. To this end, inspiration
needs to be taken from biological examples. For instance, insects are able
to optimally solve complex environment navigation problems, and many researchers
have started to mimic how these insects behave. Recent interest in
neuromorphic engineering has motivated us to present a real-time, neuromorphic,
spike-based Central Pattern Generator of application in neurorobotics,
using an arthropod-like robot. A Spiking Neural Network was designed and
implemented on SpiNNaker. The network models a complex, online-change
capable Central Pattern Generator which generates three gaits for a hexapod
robot locomotion. Recon gurable hardware was used to manage both
the motors of the robot and the real-time communication interface with the
Spiking Neural Networks. Real-time measurements con rm the simulation
results, and locomotion tests show that NeuroPod can perform the gaits
without any balance loss or added delay.Ministerio de Economía y Competitividad TEC2016-77785-
PySpike - A Python library for analyzing spike train synchrony
Understanding how the brain functions is one of the biggest challenges of our
time. The analysis of experimentally recorded neural firing patterns (spike
trains) plays a crucial role in addressing this problem. Here, the PySpike
library is introduced, a Python package for spike train analysis providing
parameter-free and time-scale independent measures of spike train synchrony. It
allows to compute similarity and dissimilarity profiles, averaged values and
distance matrices. Although mainly focusing on neuroscience, PySpike can also
be applied in other contexts like climate research or social sciences. The
package is available as Open Source on Github and PyPI.Comment: 7 pages, 6 figure
A geographically distributed bio-hybrid neural network with memristive plasticity
Throughout evolution the brain has mastered the art of processing real-world
inputs through networks of interlinked spiking neurons. Synapses have emerged
as key elements that, owing to their plasticity, are merging neuron-to-neuron
signalling with memory storage and computation. Electronics has made important
steps in emulating neurons through neuromorphic circuits and synapses with
nanoscale memristors, yet novel applications that interlink them in
heterogeneous bio-inspired and bio-hybrid architectures are just beginning to
materialise. The use of memristive technologies in brain-inspired architectures
for computing or for sensing spiking activity of biological neurons8 are only
recent examples, however interlinking brain and electronic neurons through
plasticity-driven synaptic elements has remained so far in the realm of the
imagination. Here, we demonstrate a bio-hybrid neural network (bNN) where
memristors work as "synaptors" between rat neural circuits and VLSI neurons.
The two fundamental synaptors, from artificial-to-biological (ABsyn) and from
biological-to- artificial (BAsyn), are interconnected over the Internet. The
bNN extends across Europe, collapsing spatial boundaries existing in natural
brain networks and laying the foundations of a new geographically distributed
and evolving architecture: the Internet of Neuro-electronics (IoN).Comment: 16 pages, 10 figure
Sun Sensor Based on a Luminance Spiking Pixel Array
We present a novel sun sensor concept. It is the very first sun sensor built with an address event representation spiking pixel matrix. Its pixels spike with a frequency proportional to illumination. It offers remarkable advantages over conventional digital sun sensors based on active pixel sensor (APS) pixels. Its output data flow is quite reduced. It is possible to resolve the sun position just receiving one single event operating in time-to-first-spike mode. It operates with a latency in the order of milliseconds. It has higher dynamic range than APS image sensors (higher than 100 dB). A custom algorithm to compute the centroid of the illuminated pixels is presented. Experimental results are provided.Universidad de Cádiz PR2016-072Ministerio de Economía y Competitividad TEC2015-66878-C3-1-RJunta de Andalucía TIC 2012- 2338Office of Naval Research (USA) N00014141035
Dynamic Vision Sensor integration on FPGA-based CNN accelerators for high-speed visual classification
Deep-learning is a cutting edge theory that is being applied to many fields.
For vision applications the Convolutional Neural Networks (CNN) are demanding
significant accuracy for classification tasks. Numerous hardware accelerators
have populated during the last years to improve CPU or GPU based solutions.
This technology is commonly prototyped and tested over FPGAs before being
considered for ASIC fabrication for mass production. The use of commercial
typical cameras (30fps) limits the capabilities of these systems for high speed
applications. The use of dynamic vision sensors (DVS) that emulate the behavior
of a biological retina is taking an incremental importance to improve this
applications due to its nature, where the information is represented by a
continuous stream of spikes and the frames to be processed by the CNN are
constructed collecting a fixed number of these spikes (called events). The
faster an object is, the more events are produced by DVS, so the higher is the
equivalent frame rate. Therefore, these DVS utilization allows to compute a
frame at the maximum speed a CNN accelerator can offer. In this paper we
present a VHDL/HLS description of a pipelined design for FPGA able to collect
events from an Address-Event-Representation (AER) DVS retina to obtain a
normalized histogram to be used by a particular CNN accelerator, called
NullHop. VHDL is used to describe the circuit, and HLS for computation blocks,
which are used to perform the normalization of a frame needed for the CNN.
Results outperform previous implementations of frames collection and
normalization using ARM processors running at 800MHz on a Zynq7100 in both
latency and power consumption. A measured 67% speedup factor is presented for a
Roshambo CNN real-time experiment running at 160fps peak rate.Comment: 7 page
- …