Geometry-Based Multiple Camera Head Detection in Dense Crowds

This paper addresses the problem of head detection in crowded environments. Our detection is based entirely on the geometric consistency across cameras with overlapping fields of view, and no additional learning process is required. We propose a fully unsupervised method for inferring scene and camera geometry, in contrast to existing algorithms which require specific calibration procedures. Moreover, we avoid relying on the presence of body parts other than heads or on background subtraction, which have limited effectiveness under heavy clutter. We cast the head detection problem as a stereo MRF-based optimization of a dense pedestrian height map, and we introduce a constraint which aligns the height gradient according to the vertical vanishing point direction. We validate the method in an outdoor setting with varying pedestrian density levels. With only three views, our approach is able to detect simultaneously tens of heavily occluded pedestrians across a large, homogeneous area.Comment: Proceedings of the 28th British Machine Vision Conference (BMVC) - 5th Activity Monitoring by Multiple Distributed Sensing Workshop, 201

Detecting and Tracking Cells using Network Flow Programming

We propose a novel approach to automatically detecting and tracking cell populations in time-lapse images. Unlike earlier ones that rely on linking a predetermined and potentially under-complete set of detections, we generate an overcomplete set of competing detection hypotheses. We then perform detection and tracking simultaneously by solving an integer program to find an optimal and consistent subset. This eliminates the need for heuristics to handle missed detections due to occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging image sequences consisting of clumped cells and show that it outperforms state-of-the-art techniques

Simultaneous Association and Localization for Multi-Camera Multi-Target Tracking

학위논문 (박사)-- 서울대학교 대학원 공과대학 전기·컴퓨터공학부, 2017. 8. 최진영.In this dissertation, we propose two approaches for three-dimensional (3D) localizing and tracking of multiple targets by using images from multiple cameras with overlapping views. The main challenge is to solve the 3D position estimation problem and the trajectory assignment problem simultaneously. However, most of the existing methods solve these problems independently. Unlike single camera multi-target tracking, it is much more complicated to solve both problems because the relationship between cameras is also taken into consideration in multi-camera. To tackle this challenge, we present two approaches: mixed multidimensional assignment approach and variational inference approach. In the mixed multidimensional assignment approach, we formulate the data association and 3D trajectory estimation problem as the mixed optimization problem with discrete and continuous variables and suggest an alternative optimization scheme which jointly solves the two coupled problems. To handle a large solution space, we develop an efficient optimization scheme that alternates between two coupled problems with a reasonable computational load. In this optimization formulation, we design a new cost function that describes 3D physical properties of each target. In the variational inference approach, we establish a maximum a posteriori (MAP) problem over trajectory assignments and 3D positions for given detections from multiple cameras. To find a solution, we develop an expectation-maximization scheme, where the probability distributions are designed by following the Boltzmann distribution of seven terms induced from multi-camera tracking settings.1 Introduction 1 1.1 Background & Challenges 1 1.2 Related Works 4 1.3 Problem Statements & Contributions 8 2 Mixed Multidimensional Assignment Approach 12 2.1 Problem Formulation 12 2.1.1 Problem Statements 12 2.1.2 Cost Design 17 2.2 Optimization 22 2.2.1 Spatio-temporal Data Association 23 2.2.2 3D Trajectory Estimation 31 2.2.3 Initialization 33 2.3 Application: Real-time 3D localizing and tracking system 35 2.3.1 System overview 36 2.3.2 Detection 37 2.3.3 Tracking 39 2.4 Appendix 42 2.4.1 Derivation of equation (2.35) 42 3 Variational Inference Approach 44 3.1 Problem Formulation 44 3.1.1 Notations 44 3.1.2 MAP formulation 46 3.2 Optimization 48 3.2.1 Posterior distribution 48 3.2.2 V-EM algorithm 51 3.3 Appendix 56 3.3.1 Derivation of equation (3.12) 56 3.3.2 Derivation of equation (3.27-3.32) 56 3.3.3 Deriving optimal mean and covariance matrix (3.33-3.35) 59 3.3.4 Definition of A and b in (3.22) 62 4 Experiments 63 4.1 Datasets 63 4.1.1 PETS 2009 63 4.1.2 PSN-University 64 4.2 Evaluation Metrics 66 4.3 Results and Discussion 67 4.3.1 Mixed Multidimensional Assignment Approach 67 4.3.2 Variational Inference Approach 82 4.3.3 Comparisons of Two Approaches 93 5 Conclusion 98 5.1 Concluding Remarks 98 5.2 Future Work 99 Abstract (In Korean) 112Docto

Joint Multi-person Detection and Tracking from Overlapping Cameras

We present a system to track the positions of multiple persons in a scene from overlapping cameras. The distinguishing aspect of our method is a novel, two-step approach that jointly estimates person position and track assignment. The proposed approach keeps solving the assignment problem tractable, while taking into account how different assignments influence feature measurement. In a hypothesis generation stage, the similarity between a person at a particular position and an active track is based on a subset of cues (appearance, motion) that are guaranteed observable in the camera views. This allows for efficient computation of the K-best joint estimates for person position and track assignment under an approximation of the likelihood function. In a subsequent hypothesis verification stage, the known person positions associated with these K-best solutions are used to define a larger set of actually visible cues, which enables a re-ranking of the found assignments using the full likelihood function. We demonstrate that our system outperforms the state-of-the-art on four challenging multi-person datasets (indoor and outdoor), involving 3-5 overlapping cameras and up to 23 persons simultaneously. Two of these datasets are novel: we make the associated images and annotations public to facilitate benchmarking

Tracking Interacting Objects in Image Sequences

Object tracking in image sequences is a key challenge in computer vision. Its goal is to follow objects that move or evolve over time while preserving the identity of each object. However, most existing approaches focus on one class of objects and model only very simple interactions, such as the fact that different objects do not occupy the same spatial location at a given time instance. They ignore that objects may interact in more complex ways. For example, in a parking lot, a person may get in a car and become invisible in the scene. In this thesis, we focus on tracking interacting objects in image sequences. We show that by exploiting the relationship between different objects, we can achieve more reliable tracking results. We explore a wide range of applications, such as tracking players and the ball in team sports, tracking cars and people in a parking lot and tracking dividing cells in biomedical imagery. We start by tracking the ball in team sports, which is a very challenging task because the ball is often occluded by the players. We propose a sequential approach that tracks the players first, and then tracks the ball by deciding which player, if any, is in possession of the ball at any given time. This is very different from standard approaches that first attempt to track the ball and only then to assign possession. We show that our method substantially increases performance when applied to long basketball and soccer sequences. We then focus on simultaneously tracking interacting objects. We achieve this by formulating the tracking problem as a network-flow Mixed Integer Program, and expressing the fact that one object can appear or disappear at locations of another in terms of linear flow constraints. We demonstrate our method on scenes involving cars and passengers, bags being carried and dropped by people, and balls being passed from one player to the next in team sports. In particular, we show that by estimating jointly and globally the trajectories of different types of objects, the presence of the ones which were not initially detected based solely on image evidence can be inferred from the detections of the others. We finally extend our approach to dividing cells in biomedical imagery. In this case, cells interact by overlapping with each other and giving birth to daughter cells. We propose a novel approach to automatically detecting and tracking cell populations in time-lapse images. Unlike earlier approaches that rely on linking a predetermined and potentially incomplete set of detections, we generate an overcomplete set of competing detection hypotheses. We then perform detection and tracking simultaneously by solving an integer program to find the optimal and consistent subset. This eliminates the need for heuristics to handle missed detections due to occlusions and complex morphology. We demonstrate the effectiveness of our approach on a range of challenging image sequences consisting of clumped cells and show that it outperforms the state-of-the-art techniques