Tracking by Prediction: A Deep Generative Model for Mutli-Person localisation and Tracking

Current multi-person localisation and tracking systems have an over reliance on the use of appearance models for target re-identification and almost no approaches employ a complete deep learning solution for both objectives. We present a novel, complete deep learning framework for multi-person localisation and tracking. In this context we first introduce a light weight sequential Generative Adversarial Network architecture for person localisation, which overcomes issues related to occlusions and noisy detections, typically found in a multi person environment. In the proposed tracking framework we build upon recent advances in pedestrian trajectory prediction approaches and propose a novel data association scheme based on predicted trajectories. This removes the need for computationally expensive person re-identification systems based on appearance features and generates human like trajectories with minimal fragmentation. The proposed method is evaluated on multiple public benchmarks including both static and dynamic cameras and is capable of generating outstanding performance, especially among other recently proposed deep neural network based approaches.Comment: To appear in IEEE Winter Conference on Applications of Computer Vision (WACV), 201

Detect-and-Track: Efficient Pose Estimation in Videos

This paper addresses the problem of estimating and tracking human body keypoints in complex, multi-person video. We propose an extremely lightweight yet highly effective approach that builds upon the latest advancements in human detection and video understanding. Our method operates in two-stages: keypoint estimation in frames or short clips, followed by lightweight tracking to generate keypoint predictions linked over the entire video. For frame-level pose estimation we experiment with Mask R-CNN, as well as our own proposed 3D extension of this model, which leverages temporal information over small clips to generate more robust frame predictions. We conduct extensive ablative experiments on the newly released multi-person video pose estimation benchmark, PoseTrack, to validate various design choices of our model. Our approach achieves an accuracy of 55.2% on the validation and 51.8% on the test set using the Multi-Object Tracking Accuracy (MOTA) metric, and achieves state of the art performance on the ICCV 2017 PoseTrack keypoint tracking challenge.Comment: In CVPR 2018. Ranked first in ICCV 2017 PoseTrack challenge (keypoint tracking in videos). Code: https://github.com/facebookresearch/DetectAndTrack and webpage: https://rohitgirdhar.github.io/DetectAndTrack

Hippocampus and retrosplenial cortex combine path integration signals for successful navigation

The current study used fMRI in humans to examine goal-directed navigation in an open field environment. We designed a task that required participants to encode survey-level spatial information and subsequently navigate to a goal location in either first person, third person, or survey perspectives. Critically, no distinguishing landmarks or goal location markers were present in the environment, thereby requiring participants to rely on path integration mechanisms for successful navigation. We focused our analysis on mechanisms related to navigation and mechanisms tracking linear distance to the goal location. Successful navigation required translation of encoded survey-level map information for orientation and implementation of a planned route to the goal. Our results demonstrate that successful first and third person navigation trials recruited the anterior hippocampus more than trials when the goal location was not successfully reached. When examining only successful trials, the retrosplenial and posterior parietal cortices were recruited for goal-directed navigation in both first person and third person perspectives. Unique to first person perspective navigation, the hippocampus was recruited to path integrate self-motion cues with location computations toward the goal location. Last, our results demonstrate that the hippocampus supports goal-directed navigation by actively tracking proximity to the goal throughout navigation. When using path integration mechanisms in first person and third person perspective navigation, the posterior hippocampus was more strongly recruited as participants approach the goal. These findings provide critical insight into the neural mechanisms by which we are able to use map-level representations of our environment to reach our navigational goals