26 research outputs found
The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
New vision sensors, such as the Dynamic and Active-pixel Vision sensor
(DAVIS), incorporate a conventional global-shutter camera and an event-based
sensor in the same pixel array. These sensors have great potential for
high-speed robotics and computer vision because they allow us to combine the
benefits of conventional cameras with those of event-based sensors: low
latency, high temporal resolution, and very high dynamic range. However, new
algorithms are required to exploit the sensor characteristics and cope with its
unconventional output, which consists of a stream of asynchronous brightness
changes (called "events") and synchronous grayscale frames. For this purpose,
we present and release a collection of datasets captured with a DAVIS in a
variety of synthetic and real environments, which we hope will motivate
research on new algorithms for high-speed and high-dynamic-range robotics and
computer-vision applications. In addition to global-shutter intensity images
and asynchronous events, we provide inertial measurements and ground-truth
camera poses from a motion-capture system. The latter allows comparing the pose
accuracy of ego-motion estimation algorithms quantitatively. All the data are
released both as standard text files and binary files (i.e., rosbag). This
paper provides an overview of the available data and describes a simulator that
we release open-source to create synthetic event-camera data.Comment: 7 pages, 4 figures, 3 table
Semi-Dense 3D Reconstruction with a Stereo Event Camera
Event cameras are bio-inspired sensors that offer several advantages, such as
low latency, high-speed and high dynamic range, to tackle challenging scenarios
in computer vision. This paper presents a solution to the problem of 3D
reconstruction from data captured by a stereo event-camera rig moving in a
static scene, such as in the context of stereo Simultaneous Localization and
Mapping. The proposed method consists of the optimization of an energy function
designed to exploit small-baseline spatio-temporal consistency of events
triggered across both stereo image planes. To improve the density of the
reconstruction and to reduce the uncertainty of the estimation, a probabilistic
depth-fusion strategy is also developed. The resulting method has no special
requirements on either the motion of the stereo event-camera rig or on prior
knowledge about the scene. Experiments demonstrate our method can deal with
both texture-rich scenes as well as sparse scenes, outperforming
state-of-the-art stereo methods based on event data image representations.Comment: 19 pages, 8 figures, Video: https://youtu.be/Qrnpj2FD1e
Event-Based Visual Odometry on Non-Holonomic Ground Vehicles
Despite the promise of superior performance under challenging conditions,
event-based motion estimation remains a hard problem owing to the difficulty of
extracting and tracking stable features from event streams. In order to
robustify the estimation, it is generally believed that fusion with other
sensors is a requirement. In this work, we demonstrate reliable, purely
event-based visual odometry on planar ground vehicles by employing the
constrained non-holonomic motion model of Ackermann steering platforms. We
extend single feature n-linearities for regular frame-based cameras to the case
of quasi time-continuous event-tracks, and achieve a polynomial form via
variable degree Taylor expansions. Robust averaging over multiple event tracks
is simply achieved via histogram voting. As demonstrated on both simulated and
real data, our algorithm achieves accurate and robust estimates of the
vehicle's instantaneous rotational velocity, and thus results that are
comparable to the delta rotations obtained by frame-based sensors under normal
conditions. We furthermore significantly outperform the more traditional
alternatives in challenging illumination scenarios. The code is available at
\url{https://github.com/gowanting/NHEVO}.Comment: Accepted by 3DV 202
A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
Event-based cameras are ideal for line-based motion estimation, since they predominantly respond to edges in the scene. However, accurately determining the camera displacement based on events continues to be an open problem. This is because line feature extraction and dynamics estimation are tightly coupled when using event cameras, and no precise model is currently available for describing the complex structures generated by lines in the space-time volume of events. We solve this problem by deriving the correct non-linear parametrization of such manifolds, which we term eventails, and demonstrate its application to eventbased linear motion estimation, with known rotation from an Inertial Measurement Unit. Using this parametrization, we introduce a novel minimal 5-point solver that jointly estimates line parameters and linear camera velocity projections, which can be fused into a single, averaged linear velocity when considering multiple lines. We demonstrate on both synthetic and real data that our solver generates more stable relative motion estimates than other methods while capturing more inliers than clustering based on spatiotemporal planes. In particular, our method consistently achieves a 100% success rate in estimating linear velocity where existing closed-form solvers only achieve between 23% and 70%. The proposed eventails contribute to a better understanding of spatio-temporal event-generated geometries and we thus believe it will become a core building block of future event-based motion estimation algorithms
A 5-Point Minimal Solver for Event Camera Relative Motion Estimation
Event-based cameras are ideal for line-based motion estimation, since they
predominantly respond to edges in the scene. However, accurately determining
the camera displacement based on events continues to be an open problem. This
is because line feature extraction and dynamics estimation are tightly coupled
when using event cameras, and no precise model is currently available for
describing the complex structures generated by lines in the space-time volume
of events. We solve this problem by deriving the correct non-linear
parametrization of such manifolds, which we term eventails, and demonstrate its
application to event-based linear motion estimation, with known rotation from
an Inertial Measurement Unit. Using this parametrization, we introduce a novel
minimal 5-point solver that jointly estimates line parameters and linear camera
velocity projections, which can be fused into a single, averaged linear
velocity when considering multiple lines. We demonstrate on both synthetic and
real data that our solver generates more stable relative motion estimates than
other methods while capturing more inliers than clustering based on
spatio-temporal planes. In particular, our method consistently achieves a 100%
success rate in estimating linear velocity where existing closed-form solvers
only achieve between 23% and 70%. The proposed eventails contribute to a better
understanding of spatio-temporal event-generated geometries and we thus believe
it will become a core building block of future event-based motion estimation
algorithms