최대 가중 클릭 문제의 동적 생성법을 이용한 온라인 다중 카메라 다중 물체 추적 기법

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2016. 8. 최진영.In this dissertation, we propose an online and real-time algorithm for tracking of multiple targets with multiple cameras that have overlapping field of views. Because of its applicability, multiple target tracking with a visual sensor has been studied intensively during the recent decades. Especially, algorithms using multiple overlapping cameras have been proposed to overcome the occlusion and missing problem of target that cannot be resolved by a single camera. Since the multiple camera multiple target tracking (MCMTT) problem is more complicated than the single camera multiple target tracking (SCMTT) problem, most of MCMTT algorithms are based on a batch process which considers a whole sequence at a time. Although the batch-based algorithms have been achieved the robust performance, their usability is limited because many practical applications need an instantaneous result. The objective of this dissertation is to develop an online MCMTT algorithm that has compatible tracking performance compared to the batch-based algorithms, but requires a small amount of computations. The proposed algorithm generates track hypotheses (or simply called `track') with all possible data associations between object detections from multiple cameras through frames. Then, it picks a set of tracks that best describes the tracking of targets. To identify a good track, the quality of each track is measured by our score function. The tracking solution is, then, a set of tracks that has the maximum total score. To get the solution, we formulate the problem of finding those track set as the maximum weighted clique problem (MWCP), which is one of the widely adopted formulations for a combinatorial problem that has the pairwise compatibility relationship among the variables. MWCP is well-known NP-complete problem and its worst-case computation time is proportional to the exponent of the number of tracks. Thus, solving MWCP is intractable because the number of candidate tracks exponentially increases when the tracking progresses. To alleviate the huge computational load, we propose an online scheme that dynamically formates multiple MWCPs with small-sized subsets of candidate tracks in every frame. The scheme is motivated by that the tracking solutions from consecutive frames are very similar because the status of each target is not abruptly changed between one frame. When we assume that a specific track set is an actual solution of the previous frame, only a small number of tracks have a possibility to become a solution track of the current frame. Thus, we can narrow down the size of candidate track set with the previous solution. However, propagating only the best solution of each frame can cause irreducible error when a wrong track set is chosen as the solution because of the tracking ambiguity. To hedge the risk of this error, we find multiple good solutions at each frame and propagate the K-best solutions among them to the next frame instead of a single solution. When the candidate tracks are updated and generated with newly obtained detections at the next frame, we generate multiple subsets of the entire candidate tracks with the K-best previous solutions. Each subset consists of candidate solution tracks with respect to each of the previous solutions, and a small-sized MWCP is formated with the subset. Then, our algorithm finds multiple solutions from each MWCP and repeats above procedures until the tracking is terminated. Even the proposed algorithm solves multiple MWCPs, it has lower computational complexity than solving a single MWCP with the entire candidate tracks because the overall computational load is mainly affected by the size of the largest MWCP. Moreover, when an instantaneous result is demanded, our algorithm finds better solution than solving a single large-sized MWCP because it finds more diverse solutions under a limited solving time. Although our dynamic formulation remarkably moderates the overall computational complexity, it is still challenging to satisfy the real-time capability of the tracking system. Thus, we apply three more strategies to reduce the computation time. First, we generate tracklets, robust fragments of a target's trajectory, at each camera and generate candidate tracks with those tracklets instead of detections. This prevents a generation of many absurd tracks. Second, we adopt a heuristic algorithm called a breakout local search (BLS) to solve each MWCP. With BLS, multiple suboptimal solutions can be found efficiently within a short time. Last, we prune the candidate tracks with a probability that is calculated with the K-best solutions. The probability represents the quality of each track with respect to the overall tracking situation instead of an individual track. Thus, utilizing this probability ensures a proper pruning of candidate tracks. In the experiments with a public benchmark dataset, our algorithm shows the compatible performance compared to the state-of-the-art batch-based MCMTT algorithms. Moreover, our algorithm shows a real-time capability by achieving a satisfactory performance within a reasonable computation time. We also conduct a self-comparison to verify our dynamic MWCP formation with respect to the tracking performance and solving time. When a sufficient number of solutions are propagated, our algorithm performs better and takes shorter time than solving a single MWCP considering the entire candidate tracks.Chapter 1 Introduction 1 1.1 Background 1 1.2 Related Works 3 1.2.1 Reconstruction-and-tracking methods 4 1.2.2 Tracking-and-reconstruction methods 6 1.2.3 Unified frameworks 7 1.3 Contents of the Research 8 1.4 Thesis Organization 11 Chapter 2 Preliminaries 13 2.1 Bayesian Tracking 14 2.1.1 Recursive Bayesian Tracking 16 2.1.2 Bayesian Tracking for Multiple Targets 17 2.1.3 Multiple Hypothesis Tracking (MHT) 19 2.2 Maximum Weighted Clique Problem (MWCP) 24 2.2.1 Clique Problems 24 2.2.2 Solving MWCP 26 2.3 Breakout Local Search (BLS) 27 2.3.1 Solution exploration 28 2.3.2 Perturbation Strategies 30 2.3.3 Initial Solution and Termination Condition 32 Chapter 3 Proposed Approach 35 3.1 Problem Statements 35 3.2 Tracklet Generation 40 3.2.1 Detection-to-tracklet Matching 43 3.2.2 Matching Score with Motion Estimation 46 3.2.3 Matching Validation 49 3.3 Track Hypothesis 51 3.3.1 Tracklet Association 51 3.3.2 Online Generation of Association Sets 55 3.3.3 Track Generation 57 3.3.4 Track Score 59 3.4 Global Hypothesis 64 3.4.1 MWCP for MCMTT 65 3.4.2 BLS for MCMTT 69 3.5 Pruning 70 3.5.1 Approximated Global Track Probability 71 3.5.2 Track Pruning Scheme 72 Chapter 4 Experiments 75 4.1 Comparison with the State-of-the-art Methods 81 4.2 Influence of Parameters 84 4.3 Score Function Analysis 87 4.4 Solving Scheme Analysis 88 4.5 Qualitative Results 90 Chapter 5 Concluding Remarks 97 5.1 Conclusions 97 5.2 Future Works 98 초록 117Docto