94 research outputs found
Simultaneous Optical Flow and Segmentation (SOFAS) using Dynamic Vision Sensor
We present an algorithm (SOFAS) to estimate the optical flow of events
generated by a dynamic vision sensor (DVS). Where traditional cameras produce
frames at a fixed rate, DVSs produce asynchronous events in response to
intensity changes with a high temporal resolution. Our algorithm uses the fact
that events are generated by edges in the scene to not only estimate the
optical flow but also to simultaneously segment the image into objects which
are travelling at the same velocity. This way it is able to avoid the aperture
problem which affects other implementations such as Lucas-Kanade. Finally, we
show that SOFAS produces more accurate results than traditional optic flow
algorithms
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.Comment: Conference on Computer Vision and Pattern Recognition Workshop
Event-Based Motion Segmentation by Motion Compensation
In contrast to traditional cameras, whose pixels have a common exposure time,
event-based cameras are novel bio-inspired sensors whose pixels work
independently and asynchronously output intensity changes (called "events"),
with microsecond resolution. Since events are caused by the apparent motion of
objects, event-based cameras sample visual information based on the scene
dynamics and are, therefore, a more natural fit than traditional cameras to
acquire motion, especially at high speeds, where traditional cameras suffer
from motion blur. However, distinguishing between events caused by different
moving objects and by the camera's ego-motion is a challenging task. We present
the first per-event segmentation method for splitting a scene into
independently moving objects. Our method jointly estimates the event-object
associations (i.e., segmentation) and the motion parameters of the objects (or
the background) by maximization of an objective function, which builds upon
recent results on event-based motion-compensation. We provide a thorough
evaluation of our method on a public dataset, outperforming the
state-of-the-art by as much as 10%. We also show the first quantitative
evaluation of a segmentation algorithm for event cameras, yielding around 90%
accuracy at 4 pixels relative displacement.Comment: When viewed in Acrobat Reader, several of the figures animate. Video:
https://youtu.be/0q6ap_OSBA
Event-based Motion Segmentation with Spatio-Temporal Graph Cuts
Identifying independently moving objects is an essential task for dynamic
scene understanding. However, traditional cameras used in dynamic scenes may
suffer from motion blur or exposure artifacts due to their sampling principle.
By contrast, event-based cameras are novel bio-inspired sensors that offer
advantages to overcome such limitations. They report pixelwise intensity
changes asynchronously, which enables them to acquire visual information at
exactly the same rate as the scene dynamics. We develop a method to identify
independently moving objects acquired with an event-based camera, i.e., to
solve the event-based motion segmentation problem. We cast the problem as an
energy minimization one involving the fitting of multiple motion models. We
jointly solve two subproblems, namely event cluster assignment (labeling) and
motion model fitting, in an iterative manner by exploiting the structure of the
input event data in the form of a spatio-temporal graph. Experiments on
available datasets demonstrate the versatility of the method in scenes with
different motion patterns and number of moving objects. The evaluation shows
state-of-the-art results without having to predetermine the number of expected
moving objects. We release the software and dataset under an open source
licence to foster research in the emerging topic of event-based motion
segmentation
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Event Collapse in Contrast Maximization Frameworks
Contrast maximization (CMax) is a framework that provides state-of-the-art
results on several event-based computer vision tasks, such as ego-motion or
optical flow estimation. However, it may suffer from a problem called event
collapse, which is an undesired solution where events are warped into too few
pixels. As prior works have largely ignored the issue or proposed workarounds,
it is imperative to analyze this phenomenon in detail. Our work demonstrates
event collapse in its simplest form and proposes collapse metrics by using
first principles of space-time deformation based on differential geometry and
physics. We experimentally show on publicly available datasets that the
proposed metrics mitigate event collapse and do not harm well-posed warps. To
the best of our knowledge, regularizers based on the proposed metrics are the
only effective solution against event collapse in the experimental settings
considered, compared with other methods. We hope that this work inspires
further research to tackle more complex warp models.Comment: 19 pages, 8 figures, 3 table
- …