Robust and accurate people tracking is a key task in many promising computer-vision applications. One must deal with non-rigid targets in open-world scenarios, whose shape and appearance evolve over time. Targets may interact, causing partial or complete occlusions. This paper improves tracking by means of particle filtering, where occlusions are handled considering the target's predicted trajectories. Model drift is tackled by careful updating, based on the history of likelihood measures. A colour-based likelihood, computed from histogram similarity, is used. Experiments are carried out using sequences from the CAVIAR database.Peer Reviewe

Gonzàlez, Jordi

Huerta Casado, Iván

Rowe, Daniel

Villanueva, Juan J.

English

UPCommons. Portal del coneixement obert de la UPC

Robust Multiple-People Tracking UsingColour-Based Particle FiltersDaniel Rowe 1 , Ivan Huerta 1 , Jordi Gonzàlez 2 , and Juan J. Villanueva 11 Computer Vision Centre, Universitat Autònoma de Barcelona, Spain2 Institut de Robòtica i Informàtica Industrial, UPC, Barcelona, SpainAbstract. Robust and accurate people tracking is a key task in manypromising computer-vision applications. One must deal with non-rigidtargets in open-world scenarios, whose shape and appearance evolve overtime. Targets may interact, causing partial or complete occlusions. Thispaper improves tracking by means of particle ltering, where occlusionsare handled considering the target’s predicted trajectories. Model drift istackled by careful updating, based on the history o ikelihood measures.A colour-based likelihood, computed from histogram similarity, is used.Experiments are carried out using se quences from the CAVIAR database.1 IntroductionRobust and accurate people tracking is a key task in many promising computer-vision applications, such as smart video surveillance or human-computer inter-faces [6,4,1]. The interest in multiple-people tracking is also prompted by thechallenge of emulating the amazing capabilities of natural systems to detectmotion and keep lock on several moving objects simultaneously.However, serious diculties should b e expected. The system must deal withnon-rigid targets, often highly articulated and elastic, who may wear loose-ttingclothes. In open-world applications, neither the number of targets, nor their ap-pearance or shape can be specied in advance. Considerable foreground diver-sity should be taken into account. Further, both observed shape and appearanceevolve over time depending on the point of view, or on the local illumination andbackground, specially if these are uncontrolled. Finally, as the targets interact,they may group and split, causing occlusions, and changing the observed ap-pearance and shape. This paper enhances tracking by means of particle ltering(PF). A preliminary work was published in [11]. The main contributions of thepresented approach are the following:– it copes with clutter distracters by adopting a colour-based likelihood com-puted from histogram similarity. Colour information relative to the targetsurroundings is used to tune the colour histograms.– It deals with multiple targets simultaneously, paying special attention to thesampling impoverishment phenomenon. The systems scales well with thenumber of targets, avoiding the curse of dimensionality common to PF.114 D. Rowe et al.– Model drift is precluded by careful updating, based on likelihood measures,thereby ensuring proper tracking despite noisy measures, estimate errors,occlusions, and changes in illumination conditions and camera viewpoint.– Occlusions are handled considering the predicted trajectories of all targetswithin the scene and the history of likelihood measurements.The remainder of this paper is organised as follows. Section 2 covers the prob-abilistic framework and related approaches. A colour-based particle ﬁlter formultiple-target tracking is proposed in Section 3. Section 4 shows some experi-mental results, and section 5 summarises the conclusions.2 Probabilistic Tracking FrameworkThe computation of the state st given all evidence to date e1:t is called ﬁltering.The posterior pdf p (st | e1:t) can be calculated through recursive estimation:p (st | e1:t) ∝ p (et | st)︸ ︷︷ ︸∫p (st | st−1)︸ ︷︷ ︸p (st−1 | e1:t−1)︸ ︷︷ ︸dst−1.likelihood︸ ︷︷ ︸trans. model previous post.︸ ︷︷ ︸updating prediction(1)The pdf is projected forward according to the transition model, making aprediction. It is then updated in agreement with the new evidence, et. Whennon-Gaussian, non-linear distributions are involved, this problem is overcomeby simulating N i.i.d. random samples from the posterior pdf,{sit; i = 1 : N}.This leads to the particle ﬁlter approach. This works as follows: the posteriorpdf at time t − 1, p (st−1 | e1:t−1), is represented by a weighted set of samples,{sˆit−1, πit−1; i = 1 : N}. The set is re-sampled using normalised weights πit−1 asprobabilities. The temporal prior{sˆit}is obtained by applying the transitionmodel p (st | st−1) to each sample. The likelihood p (et | st) is represented byweights πit, which are then normalised. Expectations are approximated as:Ep(st|e1:t) (st) ∑Ni=1πitsˆit. (2)Although SIR methods have been widely used in recent years, they have impor-tant drawbacks [7]. Sampling impoverishment is one of the main ones: samplesare spread around several modes pointing out hypotheses in the state space, butmost of them may be spurious. Unfortunately, there is a non-negligible probabil-ity of losing modes, a low probability of recovering them and the remaining modescould be all spurious. Diﬀerent approaches have been taken in order to overcomethese and other issues. Nummiaro et al. [9] use a PF based on colour-histogramcues. However, no multiple-target tracking is considered, which implies that noscene event such as target grouping or occlusion can be analysed. Perez et al.[10] propose also a PF based on a colour-histogram likelihood. They introduceinteresting extensions in multiple-part modelling, incorporation of backgroundinformation, and multiple-target tracking. Nevertheless, it requires an extremelyRobust Multiple-People Tracking Using Colour-Based Particle Filters 115large number of samples, since one sample contains information about the state ofall targets, dramatically increasing the state dimensionality. Further, no appear-ance model updating is performed, what usually leads to target loss in dynamicscenes. Deutscher et al. [3] present an interesting approach called annealing parti-cle ﬁlter which aims to reduce the required number of samples. However, pruninghypotheses with lower likelihood could be undesirable in a cluttered environment.Contour tracking have also been explored [8], although this may be inappropri-ate if used as the only cue in crowded scenarios because of multiple occlusions.BraMBLe [5] is an appealing approach to multiple-blob tracking which modelsboth background and foreground using Mixtures of Gaussians (MoG). However,no model update is performed, there is a common foreground model for all tar-gets, and suﬀers for the curse of dimensionality, since it tackles multiple-targettracking combining information about all targets in every sample. Therefore,even though a great number of improvements have been introduced in recentyears, there is still much ground to cover.3 A Multi-target Colour-Based PFThe motion of the central point of an elliptical region is modelled using ﬁrst-order dynamics in image coordinates. The l-labelled target’s state is deﬁned asslt =(xlt,ult,wlt,qlt, ρlt, λlt)T , where components are the ellipse position, velocity,both axes, the appearance model, the occlusion status, and the expected targetlikelihood. A label l associates a speciﬁc appearance model to the correspond-ing samples, allowing multiple-target tracking. Given the high dimensionality ofimages, a feature extraction process is mandatory. In this approach, evidenceset are given by colour histograms computed at each predicted location and size.After the initialisation, no sample is generated using detection, since it wouldmask tracking misbehaviours. Thus, just tracking performances are tested bymeans of propagating hypotheses and weighting them according to evidence.Clearly, by incorporing detection, the general performance will be enhanced,providing the system with error-recovery capabilities.3.1 Transition ModelThe position, speed, and size of each sample are predicted according to:xˆi,lt = xi,lt−1 + ui,lt−1Δt + ξix,uˆi,lt = ult−1 + ξiu,wˆi,lt = wi,lt−1 + ξiw. (3)The random vectors ξix, ξiu, ξiw, sampled from WAGN processes, provide thesystem with a diversity of hypotheses. Sample likelihoods depend on sampleposition and size, but not on their speeds. Thus, if speeds were propagated con-sidering the previous speed, they would be in quasi open loop1. Thus, their values1 There would still be a weak relation, since speeds are used to predict positions, andposition errors can be measured, but a considerable delay would be introduced.116 D. Rowe et al.could become completely diﬀerent from the true values within a few frames, andan important proportion of samples would be wasted. In order to avoid thisphenomenon, the estimated target speed ult−1 at time t − 1 is fed back into theprediction of xˆi,lt .3.2 Likelihood FunctionThe likelihood function computes the pdf of image features given the state. Thetarget appearance can be represented by means of colour histograms. Histogramsare broadly used to represented human appearance, since they are claimed to beless sensitive than colour templates to rotations in depth, the camera point ofview, non-rigid targets, and partial occlusions. Thus, the l−model is given by:ql = qlk; k = 1 : K, (4)where K is the number of bins, and the probability of each feature is:qlk = ClMa=1δ (b (xa) − k), (5)where Cl is a normalisation constant required to ensure that∑Kk=1 qlk = 1, δthe Kronecker delta, {xa; a = 1 : M} the pixel locations, and b (xa) a functionthat associates the given pixel to its corresponding histogram bin. The targetdistribution at the predicted position xˆi,lt and ellipse size wˆi,lt , is given by pli,which is calculated in the same way as the model. The similarity between twohistograms can be computed using the following metric [2,9]:dB =1 − ρ (p,q), (6)where ρ (p,q) =∑Kk=1√pkqk. is the Bhattacharyya coeﬃcient. Therefore, similarhistograms have a high Bhattacharyya coeﬃcient, which should correspond tohigh sample weights. The computed metric can be mapped using a Gaussiandistribution [9], and samples are thus weighted according to:πi,lt = pet | sˆi,lt= N dB ;μ, σ2. (7)So far no background information has been used. However, tracking successdepends on how distinguishable the target is from a local environment. Thus,foreground features present also in its surroundings should be less importantfor target localisation. Here, an approach similar to [2] is adopted by using acentre-surround model to compute the background histogram rl according tothe outer region which encloses the target. Hence, the background histogram isused to compute a weight for each bin:ωk ={min(r∗krk); k = 1 : K}, (8)where r∗k is the minimum non-zero value. Thus, these weights are then appliedto both model and target histograms to diminish the importance of those binswhich represent the local background.Robust Multiple-People Tracking Using Colour-Based Particle Filters 1173.3 Weight NormalisationIn a multiple-target tracking scenario, those targets whose samples exhibit lowerlikelihood are more likely to be lost, since the probability of propagating onemode is proportional to the cumulative weights of its samples. In order to avoidone target absorbing other target samples, genetic drift must be prevented. Thus,a memory term, which takes into account the number of targets being tracked,is included. Weights are normalised according to:πi,lt =πi,ltN	i=1,j=lπi,jt1L, (9)where L is the number of tracked targets. Each weight is normalised accordingto the total weight of the target’s samples. Thus, all targets have the sameprobability of being propagated, since the addition of the weights of each targetsamples sums 1L . This allows multiple-target tracking using a single PF, despitethe diﬀerences between their likelihoods and the genetic drift phenomenon.3.4 State EstimationThe l-target estimates are computed according to:xlt = (1 − αx)xlt−1 + ult−1Δt+ αxLNi=1πi,lt xˆi,lt,ult = (1 − αu)ult−1 + αuxlt − xlt−1Δt,wlt = (1 − αw)wlt−1 + αwLNi=1πi,lt wi,lt, (10)where αx, αu, αw ∈ [0, 1] denote the adaptation rates. Target speeds are notestimated according to sample speeds and their weights, since signiﬁcant errorswould be introduced: samples are chosen only because of sample weights, whichdo not directly depend on the current speed. This fact could imply a signif-icant amount of jitter and many samples would be wasted. Therefore, targetspeeds are computed from successive position estimates. Further, both posi-tion and speed estimates are enhanced by regularising them according to theirhistories.The target appearance must also be updated. However, this is a sensitive taskwhich may lead to the well-known model drift phenomenon. Thus, models arethen only updated when two conditions hold: (i) the target is not occluded and(ii) the likelihood of the estimated target’s state suggests that the estimate issuﬃciently reliable. In this case, they are updated using an adaptive ﬁlter:qlt = (1 − αq)qlt−1 + αqplt, (11)where αq ∈ [0, 1] is the learning rate. In order to determine when the esti-mate is reliable, the likelihood of the current estimate is computed, p(et | slt).118 D. Rowe et al.The appearance is then updated when this value is higher than an indicator ofthe expected likelihood value, calculated following an adaptive rule:λlt = (1 − αl) λlt−1 + αlpet | slt. (12)3.5 Occlusion HandlingAlthough the appearance model is not updated during occlusions, these stillconstitute a main cause of catastrophic failures. Partial occlusions may causeinaccurate size updating, according to the area that can be seen. In case ofcomplete occlusions, sample likelihoods are meaningless, and the re-samplingphase randomly propagate them, quickly losing the target.Hence, proper handling of occlusions is crucial. The state binary variable ρlttracks the occlusion status. Occlusions are predicted according to the learneddynamics. When the predicted occlusion is signiﬁcant, and the target likelihoodis lower than the expected one given by λlt, the target state changes into occluded.Then, the following changes are introduced: (i) the adaptation rates are set tozero: neither the size, nor the velocity or the indicator itself is updated, andthe position is just propagated; (ii) those samples belonging to the occludedtarget are not re-sampled. As a result, samples are spread around the targetbecause of the uncertainty predictions terms. The other targets’ samples arere-sampled, but are not assigned to the occluded target since otherwise thisone would monopolise the whole sample set. When the occlusion is no longerpredicted or a sample likelihood exceeds the value previous to the occlusion,ρlt turns into 0, which immediately implies pruning those samples with lowerweights. Furthermore, all estimates are again updated.4 Experimental ResultsThe performance of the algorithm has been tested using the CAVIAR database2.In the sequence OneLeaveShopReenter1cor (Caviar database, 389 frames at 25fps, 384 x 288 pixels), two targets are tracked simultaneously, despite their beingarticulated and elastic objects whose dynamics are highly non-linear, and thatmove through an environment which locally mimics the target colour appearance.The ﬁrst target performs a rotation and heads towards the second one, eventuallyoccluding it. The background colour distribution is so similar to the target onesthat it constitutes a source of clutter. Furthermore, several oriented lightingsources are present, dramatically aﬀecting the target appearance. Signiﬁcantspeed and size changes can also be observed.The tracker performance is shown in Fig. 1. Both targets’ appearance modelsare updated when reliable measures are obtained, see Fig. 1.(a). Poor locali-sations and occlusions are correctly detected, thereby avoiding re-sampling ofsamples of the occluded target and erroneous dynamic and appearance modelsupdating, see Fig. 1.(b), (c). The tracker successfully recovers from occlusion, see2 http://homepages.inf.ed.ac.uk/rbf/CAVIARRobust Multiple-People Tracking Using Colour-Based Particle Filters 119(a) Frame 4: updating (b) Frame 62: tracking(c) Frame 74: occluded (d) Frame 90: recoveryFig. 1. Each target’s estimated position is denoted by an ellipse and tagged accordingly;milestones are placed on the target trajectory every 25 frames; each predicted sampleis drawn using a dark dot, whereas re-sampled particles are drawn in a light onesFig. 2. Likelihood evolutionFig. 1.(d). The maximum sample and target likelihoods, and the likelihood indi-cator is shown in Fig. 2. The tracker deals with multiple-target tracking whosedynamics are highly non-linear, despite using a simple constant speed approach.They move through an environment which mimics the target appearances. Fur-thermore, their trajectories intersect causing a severe partial occlusion. It copeswith sizeable appearance and shape changes.120 D. Rowe et al.5 ConclusionsWith this work we attempt to take a step towards solving the numerous diﬃ-culties which appear in unconstrained tracking applications. A robust likelihoodfunction is used to properly evaluate samples associated to targets which presenta high appearance variability. We rely on the Bhattacharyya coeﬃcient betweencolour histograms to perform this task. Model updating is carried out with spe-cial care, thereby overcoming the model drift phenomenon. A multiple-targettracking scenario causes several problems, including sampling impoverishmentand mutual occlusions. These issues are tackled by redeﬁning the weight nor-malisation and predicting and handling occlusions.The tracker has been successfully tested despite the fact that no detection isever used after initialisation. Future research will be focused on careful feature se-lection in order to maximise the distance between the histograms correspondingto the diﬀerent targets and the background, thereby enhancing the disambigua-tion of targets from clutter.Acknowledgements. This work has been supported by the Catalan ResearchAgency (AGAUR), by the Spanish Ministry of Education (MEC) under projectsTIC2003-08865 and DPI-2004-5414, and by the EC grant IST-027110 under theHERMES project.References1. Collins, R., Lipton, A., Kanade, T.: A System for Video Surveillance and Monitor-ing. In: 8th ITMRRS, Pittsburgh, USA, pp. 1–15. ANS (1999)2. Comaniciu, D., Ramesh, V., Meer, P.: Kernel-based Object Tracking. PAMI 25(5),564–577 (2003)3. Deutscher, J., Reid, I.: Articulated Body Motion Capture by Stochastic Search.IJCV 61(2), 185–205 (2005)4. Haritaoglu, I., Harwood, D., Davis, L.: W4: real-time surveillance of people andtheir activities. PAMI 22(8), 809–830 (2000)5. Isard, M., MacCormick, J.: BraMBLe: A Bayesian Multiple-Blob Tracker. In: 8thICCV, Vancouver, Canada, vol. 2, pp. 34–41. IEEE, Nagoya, Japan (2001)6. Kahn, R., Swain, M., Prokopowicz, P., Firby, R.: Gesture Recognition Using thePerseus Architecture. In: CVPR, San Francisco, USA, pp. 734–741. IEEE, Nagoya,Japan (1996)7. King, O., Forsyth, D.: How Does CONDENSATION Behave with a Finite Numberof Samples? 6th ECCV, Ireland 1, 695–709 (2000)8. MacCormick, J., Blake, A.: A Probabilistic Exclusion Principle for Tracking Mul-tiple Objects. IJCV 39(1), 57–71 (2000)9. Nummiaro, K., Koller-Meier, E., Van Gool, L.: An Adaptive Color-Based ParticleFilter. IVC 21(1), 99–110 (2003)10. Pérez, P., Hue, C., Vermaak, J., Gangnet, M.: Color-based Probabilistic Tracking.7th ECCV, Copenhaguen, Denmark. LNCS, pp. 661–675. Springer, Heidelberg(2002)11. Rowe, D., Rius, I., González, J., Villanueva, J.J.: Improving Tracking by HandlingOcclusions. 3rd ICAPR, UK. LNCS, vol. 2, pp. 384–393. Springer, Heidelberg (2005)

Robust multiple-people tracking using color-based particle filters

https://upcommons.upc.edu/bitstream/2117/2687/1/doc1.pdf

Robust multiple-people tracking using color-based particle filters

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC