15,206 research outputs found
SODFormer: Streaming Object Detection with Transformer Using Events and Frames
DAVIS camera, streaming two complementary sensing modalities of asynchronous
events and frames, has gradually been used to address major object detection
challenges (e.g., fast motion blur and low-light). However, how to effectively
leverage rich temporal cues and fuse two heterogeneous visual streams remains a
challenging endeavor. To address this challenge, we propose a novel streaming
object detector with Transformer, namely SODFormer, which first integrates
events and frames to continuously detect objects in an asynchronous manner.
Technically, we first build a large-scale multimodal neuromorphic object
detection dataset (i.e., PKU-DAVIS-SOD) over 1080.1k manual labels. Then, we
design a spatiotemporal Transformer architecture to detect objects via an
end-to-end sequence prediction problem, where the novel temporal Transformer
module leverages rich temporal cues from two visual streams to improve the
detection performance. Finally, an asynchronous attention-based fusion module
is proposed to integrate two heterogeneous sensing modalities and take
complementary advantages from each end, which can be queried at any time to
locate objects and break through the limited output frequency from synchronized
frame-based fusion strategies. The results show that the proposed SODFormer
outperforms four state-of-the-art methods and our eight baselines by a
significant margin. We also show that our unifying framework works well even in
cases where the conventional frame-based camera fails, e.g., high-speed motion
and low-light conditions. Our dataset and code can be available at
https://github.com/dianzl/SODFormer.Comment: 18 pages, 15 figures, in IEEE Transactions on Pattern Analysis and
Machine Intelligenc
CASTNet: Community-Attentive Spatio-Temporal Networks for Opioid Overdose Forecasting
Opioid overdose is a growing public health crisis in the United States. This
crisis, recognized as "opioid epidemic," has widespread societal consequences
including the degradation of health, and the increase in crime rates and family
problems. To improve the overdose surveillance and to identify the areas in
need of prevention effort, in this work, we focus on forecasting opioid
overdose using real-time crime dynamics. Previous work identified various types
of links between opioid use and criminal activities, such as financial motives
and common causes. Motivated by these observations, we propose a novel
spatio-temporal predictive model for opioid overdose forecasting by leveraging
the spatio-temporal patterns of crime incidents. Our proposed model
incorporates multi-head attentional networks to learn different representation
subspaces of features. Such deep learning architecture, called
"community-attentive" networks, allows the prediction of a given location to be
optimized by a mixture of groups (i.e., communities) of regions. In addition,
our proposed model allows for interpreting what features, from what
communities, have more contributions to predicting local incidents as well as
how these communities are captured through forecasting. Our results on two
real-world overdose datasets indicate that our model achieves superior
forecasting performance and provides meaningful interpretations in terms of
spatio-temporal relationships between the dynamics of crime and that of opioid
overdose.Comment: Accepted as conference paper at ECML-PKDD 201
- …