3 research outputs found
DART: Distribution Aware Retinal Transform for Event-based Cameras
We introduce a generic visual descriptor, termed as distribution aware
retinal transform (DART), that encodes the structural context using log-polar
grids for event cameras. The DART descriptor is applied to four different
problems, namely object classification, tracking, detection and feature
matching: (1) The DART features are directly employed as local descriptors in a
bag-of-features classification framework and testing is carried out on four
standard event-based object datasets (N-MNIST, MNIST-DVS, CIFAR10-DVS,
NCaltech-101). (2) Extending the classification system, tracking is
demonstrated using two key novelties: (i) For overcoming the low-sample problem
for the one-shot learning of a binary classifier, statistical bootstrapping is
leveraged with online learning; (ii) To achieve tracker robustness, the scale
and rotation equivariance property of the DART descriptors is exploited for the
one-shot learning. (3) To solve the long-term object tracking problem, an
object detector is designed using the principle of cluster majority voting. The
detection scheme is then combined with the tracker to result in a high
intersection-over-union score with augmented ground truth annotations on the
publicly available event camera dataset. (4) Finally, the event context encoded
by DART greatly simplifies the feature correspondence problem, especially for
spatio-temporal slices far apart in time, which has not been explicitly tackled
in the event-based vision domain.Comment: 12 pages, revision submitted to TPAMI in Nov 201
Speed Invariant Time Surface for Learning to Detect Corner Points with Event-Based Cameras
We propose a learning approach to corner detection for event-based cameras
that is stable even under fast and abrupt motions. Event-based cameras offer
high temporal resolution, power efficiency, and high dynamic range. However,
the properties of event-based data are very different compared to standard
intensity images, and simple extensions of corner detection methods designed
for these images do not perform well on event-based data. We first introduce an
efficient way to compute a time surface that is invariant to the speed of the
objects. We then show that we can train a Random Forest to recognize events
generated by a moving corner from our time surface. Random Forests are also
extremely efficient, and therefore a good choice to deal with the high capture
frequency of event-based cameras ---our implementation processes up to 1.6Mev/s
on a single CPU. Thanks to our time surface formulation and this learning
approach, our method is significantly more robust to abrupt changes of
direction of the corners compared to previous ones. Our method also naturally
assigns a confidence score for the corners, which can be useful for
postprocessing. Moreover, we introduce a high-resolution dataset suitable for
quantitative evaluation and comparison of corner detection methods for
event-based cameras. We call our approach SILC, for Speed Invariant Learned
Corners, and compare it to the state-of-the-art with extensive experiments,
showing better performance.Comment: 8 pages, 7 figures, accepted at CVPR 201
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world