In this paper we present an approach to face and object detection and recognition based on an extension of the contentbased image retrieval method of Lu and Teng (1999). The method applies vector quantisation (VQ) compression to the image stream and uses Mahalonobis weighted Euclidean distance between VQ histograms as the measure of image similarity. This distance measure retains both colour and spatial feature information but has the useful property of being relatively insensitive to changes in scale and rotation. The method is applied to real images for face recognition and face detection applications. Tracking and object detection can be coded relatively efficiently due to the data reduction afforded by VQ compression of the data stream. Additional computational efficiency is obtained through a variation of the tree structured fast VQ algorithm also presented here

Lovell, B. C.

Walder, C. J.

University of Queensland eSpace

FACE AND OBJECT RECOGNITION AND DETECTION USING COLOUR VECTOR
QUANTISATION
Christian J. Walder and Brian C. Lovell
Intelligent Real-Time Imaging and Sensing Group,
School of Information Technology and Electrical Engineering,
The University of Queensland, Australia, 4072
{walder, lovell}@itee.uq.edu.au
ABSTRACT
In this paper we present an approach to face and object de-
tection and recognition based on an extension of the content-
based image retrieval method of Lu and Teng (1999). The
method applies vector quantisation (VQ) compression to the
image stream and uses Mahalonobis weighted Euclidean
distance between VQ histograms as the measure of image
similarity. This distance measure retains both colour and
spatial feature information but has the useful property of
being relatively insensitive to changes in scale and rotation.
The method is applied to real images for face recognition
and face detection applications. Tracking and object de-
tection can be coded relatively efficiently due to the data
reduction afforded by VQ compression of the data stream.
Additional computational efficiency is obtained through a
variation of the tree structured fast VQ algorithm also pre-
sented here.
1. MOTIVATION
In recent years considerable advances have been made in
face recognition technology. In 1998 it was reported that
computers can recognize faces better than humans [6], how-
ever this result refers to the classification of well segmented
face images. The recognition of faces in a cluttered image
under varied pose is a more difficult problem. The accurate
extraction of a face from a cluttered image is widely consid-
ered to be the most difficult and important step in the face
recognition process and is the subject of extensive research
[5].
Many approaches have been taken in the design of face
detection systems. Two distinct types of approach are those
that use color and those that use spatial and edge infor-
mation. A common spatial approach to the face detection
and recognition problems is the principal components anal-
ysis method, often referred to as eigenfaces [7] in the face
recognition context. This is a template matching approach
that compares a template of the target to each part of the
image. A major shortcoming of this technique is its sensi-
tivity to rotation and scaling. Indeed, Cendrillon[8] found
that the eigenface method was effective over only a 12◦ and
12% range of rotation and scale respectively, while Lemieux
and Parizeau [10] found the acceptable range to be only 5◦
and 5%. By way of contrast, the simple colour histogram
method is both rotation and scale invariant, but also ignores
vital spatial information essential for reliable recognition
[12]. This paper presents a method which incorporates a
mixture of both color and spatial information by moving
from a scalar to a vector quantiser. We show that the pro-
posed technique preserves the desirable property of insen-
sitivity to scale and rotation of the simple colour histogram
but adds spatial sensitivity. The motivation behind this work
is to provide an additional analysis method which has differ-
ent properties from both the purely color and purely spatial
methods to provide another useful mode of analysis.
2. BACKGROUND THEORY
2.1. Vector Quantisation
Vector Quantisation (VQ) is an extension of scalar quanti-
sation to higher dimensions. VQ maps each n-dimensional
input vector to the nearest of a set of B = {mi ∈ <n : i =
1 . . . k} n-dimensional output vectors. The set B is usu-
ally referred to as the code-book and the k elements (mi) of
the code-book are referred to as code-vectors. For an input
vector, x ∈ <n, the output is the index w (the “winner”)
of the code-vector nearest to x. By “nearest” we usually
mean the closest vector in a Euclidean distance sense, thus
‖x−mw‖ ≤ ‖x−mi‖, i = 1 . . . k. Several algorithms exist
for creating a code-book that will quantise a given data set
with low distortion. The algorithm we chose for this study
is the Self Organizing Map (SOM) algorithm [2].
2.2. VQ Based Image compression
VQ can be used to achieve lossy image compression with
an arbitrary compression ratio [3]. The VQ based image
compression is achieved by representing blocks of pixels as
vectors, quantising them to the nearest codevector, and then
representing these blocks by the corresponding index into
the code-book. The compression process for an image I is
as follows:
1. A VQ code-book is generated from a suitable set of
training images using the SOM code-book generat-
ing algorithm. Each code-vector is associated with a
unique index in the code-book.
2. The image I is divided into disjoint blocks of k by k
pixels. Each pixel is represented by an RGB triple.
3. The vectors representing each of these blocks are then
quantised using the code-book of step 1. The com-
pressed form of the image consists of the indices of
the best matching code-vector for each of the blocks
in the image.
4. The input to the decoder is then the matrix of the in-
dex numbers of step 3, as well as the code-book of
step 1. The decoder retrieves the code vector corre-
sponding to each stored index number, and replaces
the corresponding block in the reconstructed image
with the code-vector.
3. IMAGE CLASSIFICATION
The “feature vector” used for the purpose of image classi-
fication is the histogram of the VQ-compressed image rep-
resentation described in Section 2.2. This is similar to what
Lu and Teng [1] used in the domain of content based image
retrieval, but we use a single code-book including colour
information whereas Lu and Teng use a separate code-book
for each color component. This difference is significant be-
cause it incorporates more spatial information pertaining to
intensity variations in the different color components. The
code-book used for the compression is generated using the
SOM algorithm, the training data being taken from example
images of the target (for example a person’s face).
For a VQ compression scheme using a block size of m
by m pixels, a 3D image matrix I of dimensions mh by
mw by 3 (m,h,w ∈ Z) will be reduced in size by VQ-
compression to a 2D matrix J of size h by w, the elements
of J being the index numbers of the corresponding code-
vectors resulting from the compression process. Denoting
the code-book B = {c1, c2, . . . , ck ∈ <3m2}, the histogram
h = {h1, h2, . . . , hk} is then computed where hi is the
number of occurrences of code-vector index i in J . This
histogram is herein referred to as the “VQ-histogram”. For
classification purposes, we use several training images of
each class of image to be classified (e.g., several training im-
ages of each person’s face) to determine the intraclass varia-
tion of the feature vector. Because some histogram bins are
more significant than others for classification purposes, we
use the Mahalanobis distance between the VQ-histograms
as follows.
We use the distance
d(D,h) =
k∑
i=1
‖mi − hi‖
σi
(1)
where h = {h1, h2, . . . , hk} is the k-bin VQ-histogram be-
ing compared to the setD of VQ-histograms of a given class
of image (e.g., a particular person’s face). The set D =
{h1, h2, . . . , hn} contains the n histograms of a particular
class, where hj = {hj1, hj2, . . . , hjk} is the j-th histogram in
the set. m = {m1,m2, . . . ,mk} and σ = {σ1, σ2, . . . , σk}
are the mean and variance of the hj in D across j = 1 . . . n.
That is, mi =
∑k
j=1
hj
i
k , etc. The motivation for the Maha-
lanobis distance is that the contribution of each bin to this
distance should be weighted inversely to the variance of that
bin across D. This is because high variance is indicative of
an unreliable descriptor.
4. OBJECT DETECTION
The object detection system is an extension of the image
classification system presented in Section 3. The object
detection system is based on the “template matching” ap-
proach, whereby a “window” is swept around the image and
at each point some similarity measure is calculated between
the windowed region and the target. The similarity metric
used here is exactly that of the image classification system -
the Mahalanobis distance of the VQ-histograms.
Before training (calculation of the Mahalanobis means
and variances) commences, the training images are scaled to
match the size of the window used — a parameter selected
by the user. For optimal performance, it would seem that
the size of the window should be similar to the expected
size of the target in the image to be scanned. Despite this
expectation, the system proves to be quite robust to scale
changes even with a fixed window size (see Section 6).
We suggest that this insensitivity to scale is probably
due to the regular nature of most images of interest which
have reasonably large regions of featureless areas of similar
colour. A simple way to understand this point is to consider
laying a ceramic tile mosaic of an image such as a persons
face. Regardless of the size of the face, as long as the tiles
were small in size compared to the face the tiler would still
use the same set of tile colours in the same proportions. It
would largely be the run lengths of same colour tiles that
would change with scale. Thus the VQ-histogram measure
is relatively insensitive to scale over a wide range.
Figure 1: An example of the intermediate steps in the face
detection process. The second image is a VQ-histogram dis-
tance map of the first image, for the face target.
The efficiency of the algorithm benefits from the fact
that the image only needs to be VQ compressed once per
frame using the fast VQ algorithm of Section 5. Following
this step, the histograms are computed on windows in this
compressed format, which decreases computational load.
Additionally, the histogram of a region A can be calcu-
lated quickly if the histogram of an overlapping region B
is known by taking the VQ-histogram of B, subtracting the
VQ-histogram of B∩ A¯ and adding the histogram of A∩ B¯.
After sweeping this window around the image and com-
puting the Mahalanobis distance at each point, these dis-
tances are stored in a two dimensional distance map. An
example distance map is depicted in Figure 1.
A user-defined threshold is applied to the distance map
and the centroids of any clusters determined. We use the
“subtractive clustering” [4] algorithm because it has the fa-
cility to locate multiple clusters without knowing in advance
how many to expect.
5. FAST VQ ALGORITHM
A well known speedup to the standard vector quantisation
algorithm is the Tree Structured VQ (TSVQ) algorithm [3].
This involves generating a VQ code-book from a set of in-
put vectors while forming a balanced tree structure to allow
for fast searching. Using this tree structure, the winning
code-vector can be approximated very quickly in compari-
son to the slower (but 100% accurate) “exhaustive” search
method. For the system being presented in this paper, how-
ever, it was desirable to retain the SOM code-book generat-
ing algorithm but then to speed up the quantisation process
through a tree indexing scheme. To achieve this the TSVQ
algorithm was adapted to produce a tree structure on an ex-
isting code-book, as presented below.
In the following, the In are the indices of C∗ that fall in
the Voronoi region1 of node n with code-book Cn. The al-
1The Voronoi region of a code-vector c is the region in which all (input)
gorithm constructs a tree in which the jth node will contain
a code-book Cj and a list of code-vector indices Ij corre-
sponding to code-vectors in C∗. Terminal nodes will have
code-books of less than s code-vectors, and non-terminal
nodes will have m code-vectors.
Given: A code-book C∗ of n code-vectors to be trans-
formed into an m-ary TSVQ structure, and a maximum ter-
minal node code-book size of s.
1. Set the index i to 1 (the root node index) and set I1 =
{1, 2, . . . , n} (these are the indices corresponding to
the code-vectors in C∗).
2. If the number of indices in Ii is greater than s then:
- Perform the SOM VQ code-book generating al-
gorithm on those code-vectors inC∗ correspond-
ing to indices in Ii. Do this with the number of
output code-vectors constrained to m, and set
Ci to the resultant code-book. Go to step 3.
Otherwise:
- Set Ci to contain all the code-vectors in C∗ cor-
responding to index numbers in Ii. (Do not go
to step 3).
3. For j in {1, 2, . . . ,m} do the following:
- Create a child node with a previously unused in-
dex, k, and with Ii as its parent. Set Ik to the
indices in Ii that correspond to the code-vectors
in C∗ that lie in the Voronoi region of the jth
code-vector of Ci. Go to step 2 with i = k (but
i stays the same here).
Determining the output code-vector for a given input
vector is then a standard tree search using the TSVQ tree
structure produced above, as detailed in [3]. Interestingly,
increasing the size of the leaf node code-books decreases
the efficiency of the algorithm, but reduces the probability
of a non-minimum quantisation distortion (the proof of this
is straightforward). This property would allow useful trade-
offs in many applications.
6. OBJECT DETECTION SYSTEM
EXPERIMENTAL RESULTS
Although many performance metrics exist for object track-
ing systems (e.g. tracking length measure [11]), these met-
rics are not absolute. Since, for this system, good conditions
can allow a perfect tracking length measure to be attained,
this type of measurement is not used. Instead, several ex-
amples are given of the output. These examples appear in
vectors are quantised to c.
Figure 2: An example of the object detection system output.
The sequence demonstrates the capacity of the system to
handle rotation and scaling of the target.
Figure 2, where the square box represents the position of the
target (the shown person’s face) as determined by the object
detection system. Additional examples of tracking multiple
targets are available on the IRIS web site [9]. These results
were obtained using 352 by 288 pixel images, which were
processed at approximately 3 frames per second on a Pen-
tium III 600MHz PC.
6.1. VQ-Histogram as a Feature Vector
It should be stressed that the VQ-histogram does not bear
enough information to replace, say, the eigenface method
of face recognition. The method does, however, have some
suitable properties for object detection — in particular tol-
erance to rotation and scale changes. The VQ-histogram is
largely a color based method, but the technique does incor-
porate some spatial information.
7. REFERENCES
[1] G. Lu and S. Teng, A Novel Image Retrieval Technique
Based on Vector Quantization, Proc. Int’l Conf. Com-
putational Intelligence for Modeling, Control and Au-
tomation, Vienna, Austria, 1999.
[2] T. Kohonen, Self-Organizing Maps, Second Extended
Edition, Springer Series in Information sciences, Vol. 2,
No. 3, 1994.
[3] A. Gersho and R. Gray, Vector Quantization and Sig-
nal compression, Kluwer International Series in Engi-
neering and Computer Science, Norwell Massachesetts,
1991, pp410-423.
[4] S. Chiu, Fuzzy Model Identification Based on Clus-
ter Estimation, Journal of Intelligent & Fuzzy Systems,
Vol. 2, No. 3, 1994.
[5] R.L. Hsu, M. Adbel-Mottaleb and A.K. Jain, Face De-
tection in color Images, to appear in IEEE Trans. PAMI,
May 2002.
[6] W.Barrett, A Survey of Face Recognition Algorithms
and Testing Results, Conf. Record of the Thirty-First
Conf. on Signals, Systems and Computers, IEEE Com-
puter Society Press, Los Alamitos, California, 1998, pp.
301-305.
[7] Turk, M., Pentland, A., Face Recognition Using Eigen-
faces,, IEEE Conference on Computer Vision and Pat-
tern Recognition, 1992.
[8] R. Cendrillon and B. Lovell, Real-time face recogni-
tion using Eigenfaces, Proc. of the SPIE International
Conference on Visual Communications and Image Pro-
cessing 2000, Perth, Australia, June 2000.
[9] IRIS, http://www.itee.uq.edu.au/˜iris/
[10] A. Lemieux and M. Parizeau, Experiments on Eigen-
faces Robustness, Proc. of the International Conference,
on Pattern Recognition (ICPR), Qubec, August 11-15,
2002.
[11] 2D-Object Tracking Based on Projection-Histograms
FORWISS-Knowledge Processing Research Group,
Bavarian Research Center for Knowledge Base Sys-
tems, Am Weichselgarten, Denmark.
[12] T. W. Yoo and I. S. Oh, A Method for Real-Time Face
Region Tracking Based on Color Histogram, Proceed-
ings of the SPIE, Acquisition, Tracking and


Face and Object Recognition and Detection Using Colour Vector Quantisation

http://espace.library.uq.edu.au/view/UQ:11250/VQconfpaperWOSPA.pdf

Face and Object Recognition and Detection Using Colour Vector Quantisation

Abstract

Similar works

Full text

Available Versions

University of Queensland eSpace