15 research outputs found
CED: Color Event Camera Dataset
Event cameras are novel, bio-inspired visual sensors, whose pixels output
asynchronous and independent timestamped spikes at local intensity changes,
called 'events'. Event cameras offer advantages over conventional frame-based
cameras in terms of latency, high dynamic range (HDR) and temporal resolution.
Until recently, event cameras have been limited to outputting events in the
intensity channel, however, recent advances have resulted in the development of
color event cameras, such as the Color-DAVIS346. In this work, we present and
release the first Color Event Camera Dataset (CED), containing 50 minutes of
footage with both color frames and events. CED features a wide variety of
indoor and outdoor scenes, which we hope will help drive forward event-based
vision research. We also present an extension of the event camera simulator
ESIM that enables simulation of color events. Finally, we present an evaluation
of three state-of-the-art image reconstruction methods that can be used to
convert the Color-DAVIS346 into a continuous-time, HDR, color video camera to
visualise the event stream, and for use in downstream vision applications.Comment: Conference on Computer Vision and Pattern Recognition Workshop
End-to-End Learning of Representations for Asynchronous Event-Based Data
Event cameras are vision sensors that record asynchronous streams of
per-pixel brightness changes, referred to as "events". They have appealing
advantages over frame-based cameras for computer vision, including high
temporal resolution, high dynamic range, and no motion blur. Due to the sparse,
non-uniform spatiotemporal layout of the event signal, pattern recognition
algorithms typically aggregate events into a grid-based representation and
subsequently process it by a standard vision pipeline, e.g., Convolutional
Neural Network (CNN). In this work, we introduce a general framework to convert
event streams into grid-based representations through a sequence of
differentiable operations. Our framework comes with two main advantages: (i)
allows learning the input event representation together with the task dedicated
network in an end to end manner, and (ii) lays out a taxonomy that unifies the
majority of extant event representations in the literature and identifies novel
ones. Empirically, we show that our approach to learning the event
representation end-to-end yields an improvement of approximately 12% on optical
flow estimation and object recognition over state-of-the-art methods.Comment: To appear at ICCV 201
Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation
Event-based cameras record an asynchronous stream of per-pixel brightness
changes. As such, they have numerous advantages over the standard frame-based
cameras, including high temporal resolution, high dynamic range, and no motion
blur. Due to the asynchronous nature, efficient learning of compact
representation for event data is challenging. While it remains not explored the
extent to which the spatial and temporal event "information" is useful for
pattern recognition tasks. In this paper, we focus on single-layer
architectures. We analyze the performance of two general problem formulations:
the direct and the inverse, for unsupervised feature learning from local event
data (local volumes of events described in space-time). We identify and show
the main advantages of each approach. Theoretically, we analyze guarantees for
an optimal solution, possibility for asynchronous, parallel parameter update,
and the computational complexity. We present numerical experiments for object
recognition. We evaluate the solution under the direct and the inverse problem
and give a comparison with the state-of-the-art methods. Our empirical results
highlight the advantages of both approaches for representation learning from
event data. We show improvements of up to 9 % in the recognition accuracy
compared to the state-of-the-art methods from the same class of methods