2 research outputs found
Neuromorphic Optical Flow and Real-time Implementation with Event Cameras
Optical flow provides information on relative motion that is an important
component in many computer vision pipelines. Neural networks provide high
accuracy optical flow, yet their complexity is often prohibitive for
application at the edge or in robots, where efficiency and latency play crucial
role. To address this challenge, we build on the latest developments in
event-based vision and spiking neural networks. We propose a new network
architecture, inspired by Timelens, that improves the state-of-the-art
self-supervised optical flow accuracy when operated both in spiking and
non-spiking mode. To implement a real-time pipeline with a physical event
camera, we propose a methodology for principled model simplification based on
activity and latency analysis. We demonstrate high speed optical flow
prediction with almost two orders of magnitude reduced complexity while
maintaining the accuracy, opening the path for real-time deployments.Comment: Accepted for IEEE CVPRW, Vancouver 2023. Personal use of this
material is permitted. Permission from IEEE must be obtained for all other
uses, in any current or future media. Copyright 2023 IEE
Neuromorphic Optical Flow and Real-time Implementation with Event Cameras
Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments