AbstractData-driven methods of simulating a crowd of virtual humans that exhibit behaviors imitating real human crowds play an important role in crowd simulation. In this paper, we propose a Bayesian framework for the extraction of real human's behaviors which exhibit interactions in their daily life using multiple fixed cameras. The described Markov chain Monte Carlo particle filter can effectively deals with interacting targets which are influenced by the proximity and behaviors of other targets. In this paper, we use a Markov random field motion prior combing with a federate filter algorithm which treats the observations discriminatorily to substantially improve the tracking of a fixed number of interacting targets. Simultaneously, we replace the traditional importance sampling step with MCMC sampling step to get over the vast computational requirements for large numbers of targets. i.e., we focus on the data fusion and the behavior recognition process. Finally, experimental results demonstrate that the proposed Bayesian framework deals efficiently and effectively with extractions of interacting behavior

Li, Meng

Liang, Jia-Hong

Elsevier - Publisher Connector 

Behavior Extraction from Examples Using Federate MCMC-Based Particle Filtering 

 Procedia Engineering  16 ( 2011 )  291 – 297 
1877-7058 © 2011 Published by Elsevier Ltd.
doi: 10.1016/j.proeng.2011.08.1086 
Available online at www.sciencedirect.com
International Workshop on Automobile, Power and Energy Engineering
Behavior Extraction from Examples
Using Federate MCMC-Based Particle Filtering
Meng Lia, Jia-Hong Liangb,a*
a,bCollege of Mechanical Engineering and Automation, National University of Defence technology, Changsha 410073 , P.R.China
Abstract
Data-driven methods of simulating a crowd of virtual humans that exhibit behaviors imitating real human crowds
play an important role in crowd simulation. In this paper, we propose a Bayesian framework for the extraction of real
human’s behaviors which exhibit interactions in their daily life using multiple fixed cameras. The described Markov
chain Monte Carlo particle filter can effectively deals with interacting targets which are influenced by the proximity
and behaviors of other targets. In this paper, we use a Markov random field motion prior combing with a federate
filter algorithm which treats the observations discriminatorily to substantially improve the tracking of a fixed number
of interacting targets. Simultaneously, we replace the traditional importance sampling step with MCMC sampling
step to get over the vast computational requirements for large numbers of targets. i.e., we focus on the data fusion and
the behavior recognition process. Finally, experimental results demonstrate that the proposed Bayesian framework
deals efficiently and effectively with extractions of interacting behavior.
© 2010 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Society for
Automobile, Power and Energy Engineering
Keywords: behavior extraction, MCMC-based  particle filter,  Markov random field motion prior, federate filter
1. Introduction
The behaviors of human crowds in the real world characterized with trajectory, velocity, and
interaction vary significantly depending on time, place, and many other factors [1]. This work is
concerned with the problem of behavior extraction from sequential images recorded by multiple cameras.
The extracted behaviors can be used to produce many desired styles in data-driven methods of simulating
the crowd. Learning human behaviors from videos is a challenging problem because the motion of each
individual is influenced by many factors the most important one of which is the interactions between
proximate targets. We assume that each target motion in the video is Markovian and targets do not overlap
with each other vertically. Our objective is to correctly detect entering and leaving targets and obtain a
* E-mail address: mengshuqin1984@163.com
Open access under CC BY-NC-ND license.
Open access under CC BY-NC-ND license.
292   Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
record of the trajectories of targets over time, maintaining a correct, unique identification for each target
throughout. Tracking multiple targets which are identical in appearance becomes significantly more
challenging when the targets interact. Methods that appropriately deal with this difficult issue are useful
for applications where many interacting targets need to be tracked over time. In particular, they have
important implications for vision-based tracking of human behaviors which can be used to improve the
realism of crowd simulation [1,2].
There is a great potential to extract the real people’s behaviors from video sequences by use of
computer vision algorithms. A number of classical approaches have been proposed to the problem of
multiple hypothesis tracker (MHT) and the joint probabilistic data association filter (JPDAF) [2]. These
multi-target tracking algorithms which including nearest neighbor tracking in [3], the MHT in [4], and the
JPDAF in [5] and [6] have been extensively used in the context of computer vision. However, the
foregoing methods do not take into account targets’ interactions. There is an abundance of literature
devoted to the PF approach to multi-object tracking (MOT). Methods using a single-object state-space
model are usually computationally inexpensive. A shortcoming of this approach is that identities and
interactions between objects can not be easily modeled in formal (and algorithmic) terms [7]. Recently, A
joint state-space model was proposed to efficiently track a fixed number of interacting objects using a
sampling method that combines a PF formulation with MCMC sampling in [2]. This approach addressed
the problem of interaction as well, by defining a pair-wise Markov Random Field (MRF) prior in the
dynamical model, which is more computationally tractable than other methods [8]. However, the
aforementioned approaches only use a fixed camera to extract the features of targets.
Although multiple object tracking has been extensively studied in computer vision, automatic behavior
extraction is still a challenge in practice. In this paper we address the problem of interacting targets by
employing federate PF on the basis of previous studies. The federated filter fused the outputs of local
filters to generate the global estimation. The behaviors of interacting targets are modeled by a motion
model based on MRF. Considering the computational complexity of federate filter, we use MCMC-based
sampling step to replace traditional importance sampling. And the observation model is represented by
binary information picked-up from background subtraction, together with the foreground and background
color features. A specific disposal to the motion model and observation model is proposed to solve the
problem of information fusion. In our algorithm, the MCMC-based filter combing MRF dynamical model
serves as the local filter, the federated filter is used to fuse outputs of all local filters, and the global filter
result is obtained. The advantages of our approach are obvious that it combines the benefits of MCMC-
based filter and federate filter, the most valuable one of which is our approach’s powerful fault detection,
isolation and recovery abilities.
The remainder of this paper is organized as follows. Section 2 reviews the work related to Bayesian
multi-object tracking combing with the dynamic model of interacting targets. Section 3 discusses the
MCMC-based sampling step. Section 4 covers, respectively, the fusion of data from multiple cameras.
Section 5 provide some remarkable experimental validations and conclusions
2. Bayesian multi-object tracking and interactive model
2.1. traditional independent PF
Bayes filter is a classical approach in tracking multiple targets. The posterior distribution
( ), | 1... ; 1...ti t jP X Z i n j m∈ ∈  is recursively updated give all observations { }1..t tj j jZ Z Z=  up to and
including time t according to (1). n is the number of targets, and m is the number of cameras.
293 Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
( ) ( ) ( ) ( )
, 1
1
, , , , 1 , 1| | | |
i t
t t t
i t j j i t i t i t i t jX
P X Z kP Z X P X X P X Z
−
−
− −
= ∫ (1)
where ( ), , 1|i t i tP X X −  is the dynamical motion model that predicts the state ,i tX of the i-th targets at
time t given the previous state
, 1i tX − .
The likelihood ( ),|tj i tP Z X denotes the observation likelihood, the probability we observed the j-th
observation given the state
, 1i tX − at t. And k is the normalization constant.
Behavior extraction is a non-linear, non-Gaussian problem, in which PFs are accomplished. The
essence of filtering distribution is that the posterior at the previous time step is approximated by a set of
weighted particles ( ){ }( ) ( ), 1 , 1, , 1,...r ri t i tX w r N− − = , where ( ),ri tX and ( ),ri tw denote the r-th sample and its
associated weight at each time-step. We can rewrite (1) as:
( ) ( ) ( )( ) ( ), , , 1 , , 1| | |t t r ri t j j i t i t i t i t
r
P X Z kP Z X w P X X
− −
≈ ∑                              (2)
using importance sampling. And for the current time-step we draw N samples ( )
,
s
i tX from a proposal
distribution
( ) ( )( ) ( ) ( ), , , 1 , , 1|s r ri t i t i t i t i t
r
X q X w P X X
− −
= ∑                                              (3)
The weights are then computed as ( )( ) ( ), ,|r t ri t j i tw P Z X∝
2.2. Dynamic model for interactive targets
Our approach to model interactive targets introduces a capable motion model, based on MRFs. The
MRF is defined on an undirected graph (V, E), of which interactive targets define the nodes. Interactions
are defined on contiguity cliques. The state of one target is influenced by the constrains placed by the
MRF prior (Fig. 1). The pair-wise interaction between two neighbours can be writed as:
( ) ( ) ( )1 , , 1 , ,
,
| | |t t i t i t i t j t
i i j E
P X X P X X X X
− −
∈
∝ ∏ ∏ (4)
Where ( )
, ,
|i t j tX X are the pair-wise interaction potentials. Since it is easier to specify the interaction
potential in the log domain, ( )
, ,
|i t j tX X  is expressed by means of the Gibbs distribution [2]:
( ) ( )
, , , ,
| exp ( , )i t j t i t j tX X g X X ∝ − (5)
where
, ,
( , )i t j tg X X is a penalty function. In the application of behaviour extraction, , ,( , )i t j tg X X
depends only on the number of pixel overlap between the targets’ body rectangle (Fig. 2).
294   Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
Figure 1  Interaction potentials. (Left : observe from horizontal camera) A pair-wise MRF is built among individual pairs.
Proximate individuals influence each other stronger than distant individuals. (Right : observe from vertical camera)
Figure 2 Overlapping individuals. (Left : observe from horizontal camera) A pair-wise MRF is built among individual pairs.
Individuals ( ,3, 4,X Xt t ) overlapping with each other are penalized. (Right : observe from vertical camera)
In [7], the observations only come from horizontal cameras. It will bring misjudgements on estimating
the overlapping region when two individuals overlap from horizontal angle of view, but the distance
between them goes beyond each other’s influence area. In [2], although the observations come from
vertical cameras, it aims at ants’ behaviors. Because the ants can overlap with each other vertically, this
phenomenon rarely happens in human beings. This paper estimates the penalty function depending on
vertical and horizontal observations, e.g., the vertical observations revise the judgement. So when we
design the federate filters, vertical observations and its’ observation model are regarded as the master
filter.
3. An MCMC-based MRF particle filter
MCMC-based PFs generate a sequence of samples from a Markov chain of which the stationary
distribution corresponds to the target distribution after an adequate time [7]. In fact, this section pays
attention to replacing the inefficient importance sampling step of a straightforward PF implementation by
a more efficient MCMC sampling step, which drastically improves tracking results and reduces the
computation burden. Instead of using weighted samples obtained using importance sampling, the
posterior is represented as a set of unweighted samples. Thereby, on the basis of (2) ~(4), we obtain (6):
( ) ( ) ( ) ( )( ), , , , , 1
,
| | exp ( , ) |t t rt j j i t i t j t i t i t
ri j E i
P X Z kP Z X g X X P X X
−
∈
≈ − ∑∏ ∏             (6)
A Metropolis-Hastings (MH) sampler is used to sample in MCMC techniques at each time step to
generate a set of samples from a proposal distribution density ( )* |q X X , where X and *X denote the
current and proposed states. The acceptance ration for the j-th observation is computed as:
295 Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
( ) ( )
( ) ( )
* *
*
| |
min 1, | |
t
t j t t
t
t j t t
p X Z q X X
p X Z q X X

⎛ ⎞⎜ ⎟= ⎜ ⎟⎝ ⎠
                                               (7)
In MH algorithm, if 1 ≥ , *
,i tX is accepted as the new particle. Otherwise,
*X is only accepted with
probability  , and reject otherwise. In the latter case,
,i tX is accepted as the new particle. The proposal
density q is computed as [2]:
( ) ( )* ** , , , ,
1 |   if =|
0                        otherwise
i t i t i t i t
t t
q X X X X
q X X n
Δ
− −
⎧⎪
=⎨⎪⎩
                                (8)
The step of MCMC-based PF which is familiar to many related researchers is not list in this paper.
4. Framework of data fusion
The structure of federate MCMC-based MRF particle filtering is show in Fig.3, in which vertical
observations and horizontal ones are treated discriminatorily.
t
jZ
2
tZ
1
tZ
, ,
( , )i t j tg X X
, ,
( , )i t j tg X X
, ,
( , )i t j tg X X
1 1,, tP X
2 2,, tP X
,
,j j tP X
,
,g g tP X
Fig.3 The structure of federate MCMC-based MRF PF
Federate MCMC-based MRF PF which is composed of MCMC-based MRF PF and the federated filter
is based on the nonlinear state model and the information fusion theory. First of all, MCMC-based MRF
PF is employed to estimate local filters and their estimation results are acquired. Then, local filters’
estimations are fused to acquire the system’s global optimal estimate by the master filter [9].
In the structure of Federate MCMC-based MRF PF, local filters run in parallel and accept external
sensor data from horizontal cameras. Outputs of local filters are fused via master filter which provides
feedbacks
, ,
( , )i t j tg X X to local filters. The feedbacks are closely related to the observations from
vertical cameras. Estimates yielded by the federated filter gX are globally optimal or conservatively
suboptimal [10].
In addition, our system offers a chance for user interventions which provide a putting right capacity
from users. Our user guide allows the user to refine the (possibly flawed) tracking result, for example, the
“hijacking” problem.
296   Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
5. Experimental Validation and Conclusion
We adopt observations which are binary and color measurements proposed in [7], ( ), ,,t b cj j t j tZ Z Z= .
Binary observations
,
b
j tZ  are extracted using background removal techniques. Accordingly, the color
observations
,
c
j tZ are made in HS space. The observation likelihood is defined as:
( ) ( ), , ,| | , ( | )t c b bj t j t j t t j t tp Z X p Z Z X p Z X=                                 (9)
For an image sequence divided into foreground and background pixels, the observations can be
expressed as ( ), , ,,F Bj t j t j tZ Z Z= ,  then ( ) ( ) ( ), , , ,, , , , , ,| , | , | ,c b c F b F c B b Bj t j t t j t j t t j t j t tp Z Z X p Z Z X p Z Z X=
and , ,
, , ,
( | ) ( | ) ( | )b b F b Bj t t j t t j t tp Z X p Z X p Z X= .
Table 1 is the experimental results which are compared with MCMC-based MRF PF algorithms.
Table 1.  experimental results
Tracker samples failures Per target error in pixel
MCMC 1000 29 2.01 ± 0.82
MCMC 2000 26 1.86 ± 0.94
federate MCMC-based MRF PF 1000 20 1.95 ± 0.73
federate MCMC-based MRF PF 2000 15 1.29 ± 0.81
The experimental results show that federate MCMC-based MRF PF not only tracks the targets in low
errors, but has the virtue of fault detection, isolation and recovery capability. So it’s feasible to solve the
problem of behavior extraction of interacting pedestrians by way of vertical and horizontal cameras..
After successfully tracking the moving pedestrian from one frame to another in video sequences, the
problem of understanding and describing people’s behaviors, especially the group interaction behaviors,
follows naturally. Behavior understanding involves the analysis and recognition of motion patterns. The
high-level description of behaviors like actions and interactions involves the methods of DTW, FSM,
HMM, TDNN, natural language description of behaviors, etc [11]. If this problem is breached, the
simulation of interpersonal behavior in autonomous pedestrians will gain a great improvement.
References
[1] Lee, K. H., Choi, M. G., Hong, Q. and Lee, J.. Group Behavior from Video: A Data-Driven Approach to Crowd Simulation.
In. Proc. ACM SIGGRAPH / Eurographics Symposium on Computer Animation,2007, p. 109118.
[2] Z. Khan, T. Balch, and F. Dellaert. An MCMC-based particle filter for tracking multiple interacting targets. In Proc. ECCV,
Prague, May 2004.
[3] Deriche, R., Faugeras, O.: Tracking line segments. Image and Vision Computing 8,1990, p. 261–270
[4] Cox, I., Leonard, J.: Modeling a dynamic environment using a Bayesian multiple hypothesis approach. Artificial Intelligence
66,1994, p. 311–344
[5] Rasmussen, C., Hager, G.: Probabilistic data association methods for tracking complex visual objects. PAMI 23,2001, p.
560–576
[6] Schulz, D., Burgard, W., Fox, D., Cremers., A.B.: Tracking multiple moving targets with a mobile robot using particle filters
and statistical data association. In: IEEE Intl. Conf. on Robotics and Automation (ICRA),2001.
[7] Smith, K., Gatica-Perez, D., Odobez, J.M.. Using Particles to Track Varying Numbers of Interacting People. Computer
Society Conference on Computer Vision and Pattern Recognition, 2005
[8] J. Mac Cormick and A. Blake. A probabilistic exclusion principle for tracking multiple objects. In Proc. IEEE ICCV.
297 Meng Li and Jia-Hong Liang /  Procedia Engineering  16 ( 2011 )  291 – 297 
[9] HU Hai-dong, HUANG Xian-lin, LI Ming-ming, SONG Zhuo-yue. Federated unscented particle filtering algorithm for
SINS/CNS/GPS system. J.Cent.South Univ.Technol,2010, p. 778-785
[10] CARLSON N A.Federated square root filter for decentralized parallel processes[J].IEEE Transactions on Aerospace and
Electronic Systems,1990, p. 517-525.
[11] Wei-ming Hu, Tie-niu Tan. A Survey on Visual Surveillance of Object Motion and Behaviors. IEEE TRANSACTIONS
ON SYSTEMS, 2004, p. 334-352


file:///data/remote/core/dit/data/elsevier/pdf/57c/aHR0cDovL2FwaS5lbHNldmllci5jb20vY29udGVudC9hcnRpY2xlL3BpaS9zMTg3NzcwNTgxMTAyNTg3Mg==.pdf

Behavior Extraction from Examples Using Federate MCMC-Based Particle Filtering

Abstract

Similar works

Full text

Available Versions

Elsevier - Publisher Connector

Elsevier - Publisher Connector