2,190 research outputs found
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
End-to-End Learning of Representations for Asynchronous Event-Based Data
Event cameras are vision sensors that record asynchronous streams of
per-pixel brightness changes, referred to as "events". They have appealing
advantages over frame-based cameras for computer vision, including high
temporal resolution, high dynamic range, and no motion blur. Due to the sparse,
non-uniform spatiotemporal layout of the event signal, pattern recognition
algorithms typically aggregate events into a grid-based representation and
subsequently process it by a standard vision pipeline, e.g., Convolutional
Neural Network (CNN). In this work, we introduce a general framework to convert
event streams into grid-based representations through a sequence of
differentiable operations. Our framework comes with two main advantages: (i)
allows learning the input event representation together with the task dedicated
network in an end to end manner, and (ii) lays out a taxonomy that unifies the
majority of extant event representations in the literature and identifies novel
ones. Empirically, we show that our approach to learning the event
representation end-to-end yields an improvement of approximately 12% on optical
flow estimation and object recognition over state-of-the-art methods.Comment: To appear at ICCV 201
TiEV: The Tongji Intelligent Electric Vehicle in the Intelligent Vehicle Future Challenge of China
TiEV is an autonomous driving platform implemented by Tongji University of
China. The vehicle is drive-by-wire and is fully powered by electricity. We
devised the software system of TiEV from scratch, which is capable of driving
the vehicle autonomously in urban paths as well as on fast express roads. We
describe our whole system, especially novel modules of probabilistic perception
fusion, incremental mapping, the 1st and the 2nd planning and the overall
safety concern. TiEV finished 2016 and 2017 Intelligent Vehicle Future
Challenge of China held at Changshu. We show our experiences on the development
of autonomous vehicles and future trends
Event-Based Visual-Inertial Odometry on a Fixed-Wing Unmanned Aerial Vehicle
Event-based cameras are a new type of visual sensor that operate under a unique paradigm. These cameras provide asynchronous data on the log-level changes in light intensity for individual pixels, independent of other pixels\u27 measurements. Through the hardware-level approach to change detection, these cameras can achieve microsecond fidelity, millisecond latency, ultra-wide dynamic range, and all with very low power requirements. The advantages provided by event-based cameras make them excellent candidates for visual odometry (VO) for unmanned aerial vehicle (UAV) navigation. This document presents the research and implementation of an event-based visual inertial odometry (EVIO) pipeline, which estimates a vehicle\u27s 6-degrees-of-freedom (DOF) motion and pose utilizing an affixed event-based camera with an integrated Micro-Electro-Mechanical Systems (MEMS) inertial measurement unit (IMU). The front-end of the EVIO pipeline uses the current motion estimate of the pipeline to generate motion-compensated frames from the asynchronous event camera data. These frames are fed the back-end of the pipeline, which uses a Multi-State Constrained Kalman Filter (MSCKF) [1] implemented with Scorpion, a Bayesian state estimation framework developed by the Autonomy and Navigation Technology (ANT) Center at Air Force Institute of Technology (AFIT) [2]. This EVIO pipeline was tested on selections from the benchmark Event Camera Dataset [3]; and on a dataset collected, as part of this research, during the ANT Center\u27s first flight test with an event-based camera
Event Blob Tracking: An Asynchronous Real-Time Algorithm
Event-based cameras have become increasingly popular for tracking fast-moving
objects due to their high temporal resolution, low latency, and high dynamic
range. In this paper, we propose a novel algorithm for tracking event blobs
using raw events asynchronously in real time. We introduce the concept of an
event blob as a spatio-temporal likelihood of event occurrence where the
conditional spatial likelihood is blob-like. Many real-world objects generate
event blob data, for example, flickering LEDs such as car headlights or any
small foreground object moving against a static or slowly varying background.
The proposed algorithm uses a nearest neighbour classifier with a dynamic
threshold criteria for data association coupled with a Kalman filter to track
the event blob state. Our algorithm achieves highly accurate tracking and event
blob shape estimation even under challenging lighting conditions and high-speed
motions. The microsecond time resolution achieved means that the filter output
can be used to derive secondary information such as time-to-contact or range
estimation, that will enable applications to real-world problems such as
collision avoidance in autonomous driving.Comment: 17 pages, 8 figures, preprint versio
Night vision obstacle detection and avoidance based on Bio-Inspired Vision Sensors
Moving towards autonomy, unmanned vehicles rely heavily on state-of-the-art
collision avoidance systems (CAS). However, the detection of obstacles
especially during night-time is still a challenging task since the lighting
conditions are not sufficient for traditional cameras to function properly.
Therefore, we exploit the powerful attributes of event-based cameras to perform
obstacle detection in low lighting conditions. Event cameras trigger events
asynchronously at high output temporal rate with high dynamic range of up to
120 . The algorithm filters background activity noise and extracts objects
using robust Hough transform technique. The depth of each detected object is
computed by triangulating 2D features extracted utilising LC-Harris. Finally,
asynchronous adaptive collision avoidance (AACA) algorithm is applied for
effective avoidance. Qualitative evaluation is compared using event-camera and
traditional camera.Comment: Accepted to IEEE SENSORS 202
Independent Motion Detection with Event-driven Cameras
Unlike standard cameras that send intensity images at a constant frame rate,
event-driven cameras asynchronously report pixel-level brightness changes,
offering low latency and high temporal resolution (both in the order of
micro-seconds). As such, they have great potential for fast and low power
vision algorithms for robots. Visual tracking, for example, is easily achieved
even for very fast stimuli, as only moving objects cause brightness changes.
However, cameras mounted on a moving robot are typically non-stationary and the
same tracking problem becomes confounded by background clutter events due to
the robot ego-motion. In this paper, we propose a method for segmenting the
motion of an independently moving object for event-driven cameras. Our method
detects and tracks corners in the event stream and learns the statistics of
their motion as a function of the robot's joint velocities when no
independently moving objects are present. During robot operation, independently
moving objects are identified by discrepancies between the predicted corner
velocities from ego-motion and the measured corner velocities. We validate the
algorithm on data collected from the neuromorphic iCub robot. We achieve a
precision of ~ 90 % and show that the method is robust to changes in speed of
both the head and the target.Comment: 7 pages, 6 figure
- …