6,130 research outputs found
Two-Stream RNN/CNN for Action Recognition in 3D Videos
The recognition of actions from video sequences has many applications in
health monitoring, assisted living, surveillance, and smart homes. Despite
advances in sensing, in particular related to 3D video, the methodologies to
process the data are still subject to research. We demonstrate superior results
by a system which combines recurrent neural networks with convolutional neural
networks in a voting approach. The gated-recurrent-unit-based neural networks
are particularly well-suited to distinguish actions based on long-term
information from optical tracking data; the 3D-CNNs focus more on detailed,
recent information from video data. The resulting features are merged in an SVM
which then classifies the movement. In this architecture, our method improves
recognition rates of state-of-the-art methods by 14% on standard data sets.Comment: Published in 2017 IEEE/RSJ International Conference on Intelligent
Robots and Systems (IROS
A Fusion Framework for Camouflaged Moving Foreground Detection in the Wavelet Domain
Detecting camouflaged moving foreground objects has been known to be
difficult due to the similarity between the foreground objects and the
background. Conventional methods cannot distinguish the foreground from
background due to the small differences between them and thus suffer from
under-detection of the camouflaged foreground objects. In this paper, we
present a fusion framework to address this problem in the wavelet domain. We
first show that the small differences in the image domain can be highlighted in
certain wavelet bands. Then the likelihood of each wavelet coefficient being
foreground is estimated by formulating foreground and background models for
each wavelet band. The proposed framework effectively aggregates the
likelihoods from different wavelet bands based on the characteristics of the
wavelet transform. Experimental results demonstrated that the proposed method
significantly outperformed existing methods in detecting camouflaged foreground
objects. Specifically, the average F-measure for the proposed algorithm was
0.87, compared to 0.71 to 0.8 for the other state-of-the-art methods.Comment: 13 pages, accepted by IEEE TI
EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbone
Video-language pre-training (VLP) has become increasingly important due to
its ability to generalize to various vision and language tasks. However,
existing egocentric VLP frameworks utilize separate video and language encoders
and learn task-specific cross-modal information only during fine-tuning,
limiting the development of a unified system. In this work, we introduce the
second generation of egocentric video-language pre-training (EgoVLPv2), a
significant improvement from the previous generation, by incorporating
cross-modal fusion directly into the video and language backbones. EgoVLPv2
learns strong video-text representation during pre-training and reuses the
cross-modal attention modules to support different downstream tasks in a
flexible and efficient manner, reducing fine-tuning costs. Moreover, our
proposed fusion in the backbone strategy is more lightweight and
compute-efficient than stacking additional fusion-specific layers. Extensive
experiments on a wide range of VL tasks demonstrate the effectiveness of
EgoVLPv2 by achieving consistent state-of-the-art performance over strong
baselines across all downstream. Our project page can be found at
https://shramanpramanick.github.io/EgoVLPv2/.Comment: Published in ICCV 202
- …