4,839 research outputs found
Accurate Long-Term Multiple People Tracking Using Video and Body-Worn IMUs
Most modern approaches for video-based multiple people tracking rely on human appearance to exploit similarities between person detections. Consequently, tracking accuracy degrades if this kind of information is not discriminative or if people change apparel. In contrast, we present a method to fuse video information with additional motion signals from body-worn inertial measurement units (IMUs). In particular, we propose a neural network to relate person detections with IMU orientations, and formulate a graph labeling problem to obtain a tracking solution that is globally consistent with the video and inertial recordings. The fusion of visual and inertial cues provides several advantages. The association of detection boxes in the video and IMU devices is based on motion, which is independent of a person's outward appearance. Furthermore, inertial sensors provide motion information irrespective of visual occlusions. Hence, once detections in the video are associated with an IMU device, intermediate positions can be reconstructed from corresponding inertial sensor data, which would be unstable using video only. Since no dataset exists for this new setting, we release a dataset of challenging tracking sequences, containing video and IMU recordings together with ground-truth annotations. We evaluate our approach on our new dataset, achieving an average IDF1 score of 91.2%. The proposed method is applicable to any situation that allows one to equip people with inertial sensors. © 1992-2012 IEEE
Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer
Video-based human pose transfer is a video-to-video generation task that
animates a plain source human image based on a series of target human poses.
Considering the difficulties in transferring highly structural patterns on the
garments and discontinuous poses, existing methods often generate
unsatisfactory results such as distorted textures and flickering artifacts. To
address these issues, we propose a novel Deformable Motion Modulation (DMM)
that utilizes geometric kernel offset with adaptive weight modulation to
simultaneously perform feature alignment and style transfer. Different from
normal style modulation used in style transfer, the proposed modulation
mechanism adaptively reconstructs smoothed frames from style codes according to
the object shape through an irregular receptive field of view. To enhance the
spatio-temporal consistency, we leverage bidirectional propagation to extract
the hidden motion information from a warped image sequence generated by noisy
poses. The proposed feature propagation significantly enhances the motion
prediction ability by forward and backward propagation. Both quantitative and
qualitative experimental results demonstrate superiority over the
state-of-the-arts in terms of image fidelity and visual continuity. The source
code is publicly available at github.com/rocketappslab/bdmm.Comment: ICCV 202
- …