2 research outputs found
Spike-FlowNet: Event-based Optical Flow Estimation with Energy-Efficient Hybrid Neural Networks
Event-based cameras display great potential for a variety of tasks such as
high-speed motion detection and navigation in low-light environments where
conventional frame-based cameras suffer critically. This is attributed to their
high temporal resolution, high dynamic range, and low-power consumption.
However, conventional computer vision methods as well as deep Analog Neural
Networks (ANNs) are not suited to work well with the asynchronous and discrete
nature of event camera outputs. Spiking Neural Networks (SNNs) serve as ideal
paradigms to handle event camera outputs, but deep SNNs suffer in terms of
performance due to the spike vanishing phenomenon. To overcome these issues, we
present Spike-FlowNet, a deep hybrid neural network architecture integrating
SNNs and ANNs for efficiently estimating optical flow from sparse event camera
outputs without sacrificing the performance. The network is end-to-end trained
with self-supervised learning on Multi-Vehicle Stereo Event Camera (MVSEC)
dataset. Spike-FlowNet outperforms its corresponding ANN-based method in terms
of the optical flow prediction capability while providing significant
computational efficiency.Comment: European Conference on Computer Vision (ECCV) 202
ABMOF: A Novel Optical Flow Algorithm for Dynamic Vision Sensors
Dynamic Vision Sensors (DVS), which output asynchronous log intensity change
events, have potential applications in high-speed robotics, autonomous cars and
drones. The precise event timing, sparse output, and wide dynamic range of the
events are well suited for optical flow, but conventional optical flow (OF)
algorithms are not well matched to the event stream data. This paper proposes
an event-driven OF algorithm called adaptive block-matching optical flow
(ABMOF). ABMOF uses time slices of accumulated DVS events. The time slices are
adaptively rotated based on the input events and OF results. Compared with
other methods such as gradient-based OF, ABMOF can efficiently be implemented
in compact logic circuits. Results show that ABMOF achieves comparable accuracy
to conventional standards such as Lucas-Kanade (LK). The main contributions of
our paper are new adaptive time-slice rotation methods that ensure the
generated slices have sufficient features for matching,including a feedback
mechanism that controls the generated slices to have average slice displacement
within the block search range. An LK method using our adapted slices is also
implemented. The ABMOF accuracy is compared with this LK method on natural
scene data including sparse and dense texture, high dynamic range, and fast
motion exceeding 30,000 pixels per second.The paper dataset and source code are
available from http://sensors.ini.uzh.ch/databases.html.Comment: 11 pages, 10 figures, Video of result: https://youtu.be/Ss-MciioqT