1,155 research outputs found
Event-based Visual Inertial Velometer
Neuromorphic event-based cameras are bio-inspired visual sensors with
asynchronous pixels and extremely high temporal resolution. Such favorable
properties make them an excellent choice for solving state estimation tasks
under aggressive ego motion. However, failures of camera pose tracking are
frequently witnessed in state-of-the-art event-based visual odometry systems
when the local map cannot be updated in time. One of the biggest roadblocks for
this specific field is the absence of efficient and robust methods for data
association without imposing any assumption on the environment. This problem
seems, however, unlikely to be addressed as in standard vision due to the
motion-dependent observability of event data. Therefore, we propose a
mapping-free design for event-based visual-inertial state estimation in this
paper. Instead of estimating the position of the event camera, we find that
recovering the instantaneous linear velocity is more consistent with the
differential working principle of event cameras. The proposed event-based
visual-inertial velometer leverages a continuous-time formulation that
incrementally fuses the heterogeneous measurements from a stereo event camera
and an inertial measurement unit. Experiments on the synthetic dataset
demonstrate that the proposed method can recover instantaneous linear velocity
in metric scale with low latency
Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios
Event cameras are bio-inspired vision sensors that output pixel-level
brightness changes instead of standard intensity frames. These cameras do not
suffer from motion blur and have a very high dynamic range, which enables them
to provide reliable visual information during high speed motions or in scenes
characterized by high dynamic range. However, event cameras output only little
information when the amount of motion is limited, such as in the case of almost
still motion. Conversely, standard cameras provide instant and rich information
about the environment most of the time (in low-speed and good lighting
scenarios), but they fail severely in case of fast motions, or difficult
lighting such as high dynamic range or low light scenes. In this paper, we
present the first state estimation pipeline that leverages the complementary
advantages of these two sensors by fusing in a tightly-coupled manner events,
standard frames, and inertial measurements. We show on the publicly available
Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement
of 130% over event-only pipelines, and 85% over standard-frames-only
visual-inertial systems, while still being computationally tractable.
Furthermore, we use our pipeline to demonstrate - to the best of our knowledge
- the first autonomous quadrotor flight using an event camera for state
estimation, unlocking flight scenarios that were not reachable with traditional
visual-inertial odometry, such as low-light environments and high-dynamic range
scenes.Comment: 8 pages, 9 figures, 2 table
The Event-Camera Dataset and Simulator: Event-based Data for Pose Estimation, Visual Odometry, and SLAM
New vision sensors, such as the Dynamic and Active-pixel Vision sensor
(DAVIS), incorporate a conventional global-shutter camera and an event-based
sensor in the same pixel array. These sensors have great potential for
high-speed robotics and computer vision because they allow us to combine the
benefits of conventional cameras with those of event-based sensors: low
latency, high temporal resolution, and very high dynamic range. However, new
algorithms are required to exploit the sensor characteristics and cope with its
unconventional output, which consists of a stream of asynchronous brightness
changes (called "events") and synchronous grayscale frames. For this purpose,
we present and release a collection of datasets captured with a DAVIS in a
variety of synthetic and real environments, which we hope will motivate
research on new algorithms for high-speed and high-dynamic-range robotics and
computer-vision applications. In addition to global-shutter intensity images
and asynchronous events, we provide inertial measurements and ground-truth
camera poses from a motion-capture system. The latter allows comparing the pose
accuracy of ego-motion estimation algorithms quantitatively. All the data are
released both as standard text files and binary files (i.e., rosbag). This
paper provides an overview of the available data and describes a simulator that
we release open-source to create synthetic event-camera data.Comment: 7 pages, 4 figures, 3 table
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.Comment: Conference on Computer Vision and Pattern Recognition Workshop
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age
Simultaneous Localization and Mapping (SLAM)consists in the concurrent
construction of a model of the environment (the map), and the estimation of the
state of the robot moving within it. The SLAM community has made astonishing
progress over the last 30 years, enabling large-scale real-world applications,
and witnessing a steady transition of this technology to industry. We survey
the current state of SLAM. We start by presenting what is now the de-facto
standard formulation for SLAM. We then review related work, covering a broad
set of topics including robustness and scalability in long-term mapping, metric
and semantic representations for mapping, theoretical performance guarantees,
active SLAM and exploration, and other new frontiers. This paper simultaneously
serves as a position paper and tutorial to those who are users of SLAM. By
looking at the published research with a critical eye, we delineate open
challenges and new research issues, that still deserve careful scientific
investigation. The paper also contains the authors' take on two questions that
often animate discussions during robotics conferences: Do robots need SLAM? and
Is SLAM solved
Real-time Visual-Inertial Odometry for Event Cameras using Keyframe-based Nonlinear Optimization
Polymorphic conserved simple sequence repeats (SSR) markers detected among the two lowland and an upland genotypes. (DOCX 15Ă‚Â kb
Tight Fusion of Events and Inertial Measurements for Direct Velocity Estimation
Traditional visual-inertial state estimation targets absolute camera poses
and spatial landmark locations while first-order kinematics are typically
resolved as an implicitly estimated sub-state. However, this poses a risk in
velocity-based control scenarios, as the quality of the estimation of
kinematics depends on the stability of absolute camera and landmark coordinates
estimation. To address this issue, we propose a novel solution to tight
visual-inertial fusion directly at the level of first-order kinematics by
employing a dynamic vision sensor instead of a normal camera. More
specifically, we leverage trifocal tensor geometry to establish an incidence
relation that directly depends on events and camera velocity, and demonstrate
how velocity estimates in highly dynamic situations can be obtained over short
time intervals. Noise and outliers are dealt with using a nested two-layer
RANSAC scheme. Additionally, smooth velocity signals are obtained from a tight
fusion with pre-integrated inertial signals using a sliding window optimizer.
Experiments on both simulated and real data demonstrate that the proposed tight
event-inertial fusion leads to continuous and reliable velocity estimation in
highly dynamic scenarios independently of absolute coordinates. Furthermore, in
extreme cases, it achieves more stable and more accurate estimation of
kinematics than traditional, point-position-based visual-inertial odometry.Comment: Accepted by IEEE Transactions on Robotics (T-RO
- …