A spatial-color-based non-parametric background-foreground modeling strategy in a GPGPU by using CUDA is proposed. This strategy is suitable for augmented-reality applications, providing real-time high-quality results in a great variety of scenarios

Berjón Díez, Daniel

Cuevas Rodríguez, Carlos

García Santos, Narciso

Morán Burgos, Francisco

Daniel Berjon

Carlos Cuevas

Francisco Moran

Narciso Garcia

Crossref

Moving object detection strategy for augmented-reality applications in a GPGPU by using CUDA

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Moving Object Detection Strategy for Augmented-Reality 
Applications in a GPGPU by Using CUDA 
Daniel Berjón, Carlos Cuevas, Francisco Moran, and Narciso García 
theorem: 
Abstract— A spatial-color-based non-parametric background-
foreground modeling strategy in a GPGPU by using CUDA is 
proposed. This strategy is suitable for augmented-reality 
applications, providing real-time high-quality results in a great 
variety of scenarios. 
I. INTRODUCTION 
The last generation of consumer electronic devices, such as 
smart-phones, tablets or home and portable video game 
consoles, are endowed with Augmented-Reality (AR) tools 
[1]. These tools require new, fast and efficient (i.e. 
lightweight) computer vision applications [2] where moving 
object detection is a key step for several high level analysis 
tasks such as segmentation, tracking, classification, or event 
detection. 
Recently, non-parametric based moving object detection 
strategies, able to provide satisfactory results (low number of 
misdetections and false detections) in complex scenarios with 
dynamic backgrounds, non-static cameras and illumination 
changes, have been proposed [3]. These strategies, modeling 
the background and the foreground from spatial-color sets of 
reference samples, are able to improve the quality provided by 
other strategies [4]. Nevertheless, they require the evaluation 
of very large amounts of multidimensional computations and, 
thus, they are not suitable for AR applications. 
However, these strategies can greatly benefit from moving 
them to a modern programmable general-purpose graphical 
processing unit (GPGPU). These devices are becoming 
commonplace in regular computers and are even advancing 
into mobile computing scenarios [5]. They provide very good 
numeric performance and a high degree of parallelism, being 
capable of executing simultaneously hundreds or even 
thousands of threads concurrently [6]. 
In this paper we propose a novel non-parametric based 
moving object detection strategy in a GPGPU by using CUDA 
(Compute Unified Device Architecture) [7], which provides 
real-time high-quality results in a great variety of scenarios 
and, accordingly, can be used in AR applications. 
II. NON-PARAMETRIC MODELING 
Let us consider a pixel x", in the current image, defined by 
its RBG color components (Rn,Gn,Bn) and by its coordinates 
(rn,cn). The probability of x" belonging to the background, /?, 
or to the foreground, <j>, can be evaluated using Bayes' 
Pr 
•)-
Pr( <p(x"\t) 
\fi) 
where Pr(tj>) and Pr(P) are the prior probabilities for 
foreground and background, p(xn\ <j>) is the probability density 
function (pdf) estimated for the foreground and/?(x"| P) is the 
correspond pdf estimated for the background. 
Applying 5-dimensional Gaussian kernels with diagonal 
bandwidth matrices, H=diag(hR,hG,hB,hnhc), the background 
pdf for x" can be estimated, from Np reference samples, 
x¡i,i=(R¡i,i,G¡i,iB/j,i,r/j¡ic/iJ, from the 7) previous images, as: 
>(*"l/?)= 
v i 5 
Nfi(2zp\H\ sri
exp 
VA 
2h(j,j) 
and the foreground pdf can be estimated from A^ reference 
samples, z^fR^G^B^r^c^J, from the foreground pixels 
detected along the previous T¿ images as a mixture of a 
uniform function, y, and a kernel density estimation: 
p(x"\d>)= 
a-a) ^ A r (*"0-K,(/)r 
~-
ar+—, „,. . i / Z i i e x P " N¿(2z)5/2\H\/¿ -=i ;=i 2h(jj) 
where a is a mixture factor. 
To decrease the computational cost of this non-parametric 
based strategy, we propose to fix the bandwidth matrices and 
to use lookup tables (LUT). Nevertheless, for each pixel at 
each new image, a very large amount of memory accesses are 
required (typically, more than 107 samples) and, as a result, 
background and foreground modeling cannot be efficiently 
computed in typical Central Processing Units (CPUs). 
III. GPU-CUDA IMPLEMENTATION 
Our algorithm, described in Fig.l, very naturally lends itself 
to parallelization because belonging to foreground or 
background is evaluated in a per-pixel basis, which allowed us 
to devote a thread for each pixel in every stage of the process. 
New frames are transferred to the global memory of the GPU 
as textures, which allows for faster read operations. Then each 
of the steps of the algorithms is implemented in parallel stages 
as CUDA kernels, which we describe in the following 
paragraphs. 
Background model: The input image is tessellated into 
windows sized 16x16 pixels to be processed by one CUDA 
thread block each. To evaluate the sum of Gaussians on each 
pixel, we need to retrieve all the samples from past images 
belonging to the window and its neighborhood. However, 
most pixels (those inside the window) will be input to several 
Fig. 1. Memory flows for the proposed algorithm. 
threads, so we avoid reading them several times from the 
relatively slow texture memory. Pixels to be read are 
distributed evenly among the threads in the block, which 
concurrently copy them into the shared memory region, which 
is much faster. Finally, each thread computes the value of the 
sum of Gaussians for its own pixel. High arithmetic intensity 
and operands stored in a fast memory region make this 
operation very efficient. 
Foreground model: The process for this model is largely the 
same than that for the background model. However, in 
addition to the current image, this model requires the past 
foreground images. Since holding in memory both the whole 
past input images and the past foreground images does not 
make sense, the latter being a subset of the former, foreground 
images are stored as binary masks packed as 16-bit integers. 
This coding method allows the threads to disregard in a single 
comparison a whole row of the 16x16 window, greatly 
improving the speed of this phase despite the fact that the 
input spatial window is usually larger than in the background 
model. 
Classification: Only one read operation per pixel is needed, 
so the shared memory strategy is unnecessary. However, the 
image is tessellated in 256x16 windows. Thus, each thread is 
responsible for a 16x1 sub-window, yielding a whole 16-bit 
mask as described earlier, thereby eliminating the need for 
synchronization among threads. 
IV. RESULTS 
Fig. 2 depicts the quality of the obtained results in different 
scenarios containing multimodal backgrounds. These results 
Fig. 2. Quality detection analysis in different scenarios, (a) Original images, 
(b) Ground truth, (c) Detections obtained with the proposed strategy. 
show that the proposed strategy provides high-quality 
detections where moving objects are correctly identified and 
well defined. Finally, Table I resumes with the computational 
reduction resulting from the use of the parallel programming 
with CUD A. 
TABLE I 
MEAN PROCESSING TIMES (SECONDS) 
Background modeling 
Foreground modeling 
Overall strategy 
CPU 
24 
8 
32 
GPGPU 
0.15 
0.04 
0.19 
Speedup 
160x 
200x 
168x 
V. CONCLUSIONS 
Here we have described a novel background-foreground 
non-parametric based moving object detection strategy in a 
GPGPU by using CUD A. 
Obtained results have demonstrated that the proposed 
strategy is able to provide high-quality real-time results in a 
large variety of multimodal and complex scenarios and, 
therefore, it is suitable for AR tools required by the last 
generation of consumer electronic devices endowed with 
integrated smart cameras. 
REFERENCES 
[1] D. W. F. van Krevelen, and R. Poelman, "A Survey of Augmented 
Reality Technologies, Applications and Limitations", The International 
Journal of Virtual Reality, vol. 9, no. 2, pp. 1-20, 2010. 
[2] A. Mulloni, and T. Drummond, "Real-Time Detection and Tracking for 
Augmented Reality on Mobile Phones", IEEE Trans. Visualization and 
Computer Graphics, vol. 16, no. 3, pp. 355-368, 2010. 
[3] Y. Sheikh, O. Javed, and T. Kanade, "Background Subtraction for Freely 
Moving Cameras", IEEE Int. Conf. Computer Vision, pp. 1219-1225, 
2009. 
[4] Y. Sheikh, and M. Shah, "Bayesian modeling of dynamic scenes for 
object detection", IEEE Trans. Pattern Analysis and Machine 
Intelligence, vol. 27, no. 11, pp. 1778-1792, 2005. 
[5] S. P. Mohanty, "GPU-CPU Multi-Core For Real-Time Signal 
Processing", Int. Conf. Consumer Electronics, 2009. 
[6] S. F. Tsai, C. C. Cheng, C. T. Li, and L. G. Chen, "A Real-Time 1080p 
2D-to-3D Video Conversion System", IEEE Trans. Consumer 
Electronics, vol. 57, no. 2, pp. 915-922, 2011. 
[7] NVIDIA CUDA development team, "NVIDIA CUDA Compute Unified 
Device Architecture Programming Guide", NVIDIA corporation, 2007. 


Archivo Digital UPM

http://oa.upm.es/30509/1/INVE_MEM_2012_173551.pdf

Moving object detection strategy for augmented-reality applications in a GPGPU by using CUDA

Abstract

Similar works

Full text

Available Versions

Crossref

Servicio de Coordinación de Bibliotecas de la Universidad Politécnica de Madrid

Archivo Digital UPM