Ekenel, H .K.

Fischer, M.

Jiang, F.

Shi, B.

English

Feijun Jiang

Bertram Shi

Mika Fischer

Hazim K. Ekenel

Crossref

Effective discretization of Gabor features for real-time face detection

Ekenel, H . K.

KITopen

EFFECTIVE DISCRETIZATION OF GABOR FEATURES FORREAL-TIME FACE DETECTIONFeijun Jiang, Bertram ShiHong Kong University ofScience and TechnologyDepartment of ECEClear Water Bay, Kowloon, Hong KongMika Fischer, Hazım K. EkenelKarlsruhe Institute of TechnologyInstitute for AnthropomaticsAdenauerring 2, 76131 Karlsruhe, GermanyABSTRACTWe describe a real-time face detector based on Gabor fea-tures. While Gabor features often lead to improved perfor-mance, they are often avoided as they are perceived as be-ing computationally expensive. We address this in two ways.First, we propose an efficient discrete encoding method forthe Gabor feature vector. This enables us to use a computa-tionally efficient multi-stage classifier based on boosting andwinnowing. Second, we accelerate computationally complexcomputations using the parallelization provided by graphicsprocessing units (GPUs). With these innovations, the result-ing detector runs at 16.8 fps for 640 × 480 images on a PCequipped with an i5 CPU and a GTX 465 graphic card.Index Terms— Real-time, face detection, Gabor filter,boosting1. INTRODUCTIONThere have been extensive research efforts on face detection.Recently, there have been real-time face detectors proposed.Among the most popular detectors are Viola-Jones and MCTface detectors [1, 2], which use Haar wavelets or the modifiedcensus transform (MCT) as features, and boosting for classi-fication. These systems have shown very promising results.Since biological systems excel at face detection, one pos-sible way to achieve better performance would be to use a fea-ture representation that resembles what is thought to be usedby biological systems. Since the impulse responses of Gaborfilters are known to resemble the receptive fields of simplecells in the mammalian primary visual cortex, features basedon the outputs of Gabor filters have attracted much attentionfor face detection. Xiaohua et al. [3] proposed a simplifiedThis study is funded by a grant from the Germany/Hong Kong Joint Re-search Scheme sponsored by the Research Grants Council of Hong Kongand the German Academic Exchange Service (DAAD) (Reference No.G HK014/09), by the InterACT program, by the “Concept for the Future”of Karlsruhe Institute of Technology within the framework of the GermanExcellence Initiative and by OSEO, the French State agency for innovation,as part of the Quaero Program.Gabor filter that uses the subtraction of rectangle regions asan approximation of the output of Gabor filters. Adding thesesimplified Gabor features to a detector using Haar-like fea-tures and training via boosting improves the detector perfor-mance. Huang et al. [4] proposed a face detector that usesa polynomial neural network to classify Gabor features com-puted at four orientations. Kwolek [5] proposed a convolu-tional neural network face detector using Gabor filter outputsfrom two scales and two orientations. Chen et al. [6] intro-duced a hybrid face detector that cascades detectors based onGabor energy features and Haar-like features.While these studies suggest that Gabor features arepromising for face detection, they are often avoided as theyare perceived as being computationally expensive. Here, weaddress this problem by developing an effective and com-putationally simple discretization of the Gabor filter output.Previous work has typically used high dimensional continu-ous valued feature vectors defined by taking the Gabor filteroutputs or their magnitudes directly. By using a discrete en-coding, we can use a table lookup based boosting classifier.This enables us to quickly run many boosting loops, each ofwhich improves classification performance, and the results ofall boosting loops can be combined into a single lookup table.We find that the performance of our face detector exceeds thatof detectors using continuous valued Gabor filter outputs asfeatures. Thus, the loss of information due to discretizationis compensated by the increased power of the classifier pro-vided by boosting. Meanwhile it is quite efficient; it works at16.8 fps on our system with image size of 640 × 480.To examine whether discretized Gabor features are ef-fective for face detection, we first study a discretizationbased upon k-means clustering and vector quantization (VQ).While this performs well, the k-means process of searchingfor the closest center is quite time consuming. To address thisproblem, we have examined the classifiers obtained by thek-means clustering, and developed a discrete encoding calledradial phase which only requires several pixel-wise com-parison operations on the Gabor continuous outputs, whilemaintaining performance comparable to the VQ encoding.This paper is organized as follows. In Section 2, wedescribe the architecture of the classifier. In Section 3, wepresent experimental results comparing the VQ and radialphase encodings of the outputs of Gabor filters tuned to dif-ferent orientations. We also discuss the implementation ofour top performing classifier using GPU based processing.Finally, we conclude in Section 4 with a summary.2. FACE DETECTOR ARCHITECTUREThe face detector is applied to 20×20 pixel sub-images withineach image. The location of the sub-image is scanned over theentire image and at multiple scales. The scales are constructedusing an image pyramid, where each scale differs by a factorof 1.2. The detector consists of three steps: Gabor featureextraction, discrete encoding of the Gabor features and finallya multi-stage classifier. We describe these steps in more detailbelow.2.1. Gabor featuresGabor filters have an impulse response given by a 2D Gaus-sian modulated by a 2D complex exponential with spatial fre-quency parameters Ωx and Ωy , which determine the centerspatial frequency Ω =√Ωx2 + Ωy2 and orientation θ =arctan Ωx/Ωy . Here, we use a center spatial frequency (Ω)of 2π/3. The Gaussian covariance matrix is diagonal with en-tries σ2 = (1.6/Ω)2. In order to remove the DC componentin the real part of the Gabor filter, we subtract from the realpart the output of a circularly symmetric Gaussian filter withstandard deviation σ multiplied by a constant.We construct a continuous valued Gabor feature vectorby taking the real and imaginary parts of the complex valuedoutputs of N filters tuned to orientations equally spaced from0 to (N − 1)π/N , where N = 4. In order to increase therobustness of our detector against illumination variations, wenormalize each element of the Gabor feature vector by thesquare root of the sum of the energies of the responses fromall orientations.2.2. Discrete feature encodingWe examined two strategies for discretizing the normalizedfilter outputs: vector quantization of the entire feature vec-tor using a codebook obtained by k-means clustering and amethod based on discretizing each complex valued filter out-put individually based on its phase and energy.For the vector quantization encoding, we use a set of 512vectors. We generate these vectors by feeding 11,338,056normalized Gabor vectors into a k-means clustering function.Half the training vectors are taken from all 18 × 18 featurelocations of the face images in 70% of the training data. Halfare taken from non-face images, which are chosen randomlyfrom among all the negative images.−1 −0.75 −0.5 −0.25 0 0.25 0.5 0.75 1−1−0.75−0.5−0.2500.250.50.751Fig. 1. Graphical representation of the lookup table for oneclassifier in the first boosting stage. Color intensities of trian-gle/rectangle represent positive/negative weights. The greenlines show the boundaries used for the radial phase encoding.Unfortunately, the VQ encoding is computationally ex-pensive, since we must compare the Gabor feature with eachof the k vectors. Thus, we seek to leverage our knowledgeabout the structure of the feature space gleaned from design-ing the VQ based classifiers to develop a discrete encodingthat can be computed more efficiently.We define a discrete encoding that encodes the normalizedGabor filter output at each of the four orientations separately.We map each Gabor filter output to one of five indices de-pending upon which of the five regions of the complex planeshown in Fig. 1 the filter output falls into. In total, there are54 = 625 total possibilities for each Gabor feature vector.Filter outputs falling into the central circular region withradius 0.5 correspond to orientations whose Gabor filter en-ergy is smaller than the mean energy among the four orien-tations. These are less significant. The more significant filteroutputs are quantized into one of four regions, correspondingto different phases. The phase indicates the edge type, whichis necessary for face detection. Phases of 0 and π correspondto cosine modulations in the Gabor function, and thus indicatelight or dark bars at the given orientation. Phases of −π/2and π/2 correspond to sine modulations in the Gabor func-tion, and thus indicate step edges with different polarity. Byquantizing the output into four regions centered around thesevalues we ignore small position shifts of the edges. We referto this encoding as the radial phase encoding.2.3. Multi-stage classifierFor the classification, we use the same lookup table basedcascaded classifier as C. Küblbeck and A. Ernst used in theirMCT based face detector [1], which contains four cascadedstages: three Ada-boosting stages and a final winnow stage.Stages are trained sequentially. Each stage is trained onlyon samples which pass through all the previous stages. Thethreshold for comparison at each stage is set so that the truepositive detection rate is 99.5%. Negative samples are cho-sen randomly from a set of 20 × 20 sub-windows taken fromnatural images which contain no faces.For the first three stages, the classification at each stage isdetermined by the weighted sum of the outputs of a number ofweak classifiers, where the choice of the weak classifiers andtheir weighting coefficients are determined by boosting. Foreach boosting loop, each sub-image location is considered asa potential weak classifier, where we consider only the center18 × 18 (324) locations within each 20 × 20 pixel sub-imageto avoid boundary effects. The weak classifier at each sub-image location is a lookup table, where the indices correspondto the indices of the discrete encoded features and the valueof the lookup table is either 0 or 1 depending on whether theclassification result is face or non-face.We limit the first stage to use only 20 pixel locations andthe second stage to use only 80 pixel locations. The thirdstage is not limited. For each boosting stage, we run 1500boosting loops. In the initial stages, many weak classifiers ateach pixel location are used, but they can be combined into asingle lookup table.The fourth winnow stage makes use of all the locations.At each location there are separate lookup tables of weightsfor face and non-face. Initially all weights are 0, but are setto 1 the first time they are addressed during training. Theweights are updated according to the Winnow Update Rulein [1]. After training, we generate a single lookup table foreach location by subtracting the lookup tables for face andnon-face.3. EXPERIMENTS AND RESULTSFor training, we use a database of 25000 cropped face im-ages with size of 20 × 20 pixels, which includes faces withslight variations in pose angle and under different illumina-tion conditions [7]. The negative samples are from a dataset ofmore than 25000 background images. While training the firststage, the negative samples are obtained by randomly choos-ing 20×20 pixels windows from the background images. Forthe later stages, the negative samples are randomly chosenfrom the false positives that pass through all of the previousstages.We used the results of this VQ classifier to develop the ra-dial phase discrete encoding. Fig. 1 shows an example of thisanalysis with a VQ codebook size of 128. Here the final com-bined lookup table for the classifier at one sub-image locationchosen during the first boosting phase is shown graphicallyas a set of points on the complex plane. The location of eachpoint corresponds to the normalized Gabor output for the hor-izontal orientation. The triangle/rectangle encodes the signof the lookup table weight which indicates positive/negativejudgement and the intensity encodes the weight. As we can0 25 50 75 100 125 150 175 20000.10.20.30.40.50.60.7False positivesTrue positive (%)  VQ 512 centersRadial PhaseMCTMikolajaczyk et al.Viola−JonesFig. 2. The ROC curves of different detectors on FDDBset [8].see, most of the bright rectangle points lie outside the circlewith radius 0.5, while most of the bright triangle points lieinside.We benchmark these detectors using evaluation protocolfor the FDDB dataset proposed by V. Jain and E. Learned-Miller [8]. Each detector generates a ROC curve representingits performance on the FDDB benchmark dataset, which con-tains 2845 images. We compare our results with the publisheddata from the FDDB dataset, as well as the result of an MCTbased detector that uses the same multi-stage classifier and istrained with the same training data. The MCT encodes theintensities in each 3 × 3 window around a pixel location asone of 511 possible patterns.Fig. 2 shows the ROC curves of detectors based on theradial phase coding, the VQ based detector with 512 centersand the MCT baseline [1]. The performance of both the radialphase encoder and the VQ based detector exceeds that of thebaseline results published from the FDDB dataset by Mikola-jazyk et al. [9] and Viola-Jones [2], as well as that obtainedusing the MCT based detector. For example, while the truepositive is 70%, the corresponding number of false detectionsfor VQ, radial phase, MCT, Mikolajazyk et al. and Viola-Jones are 132, 141, 150, 30739 and 44147, respectively. Thisdemonstrates that appropriately chosen discretizations basedon Gabor filter outputs can provide more effective representa-tions of the image structure than the ones used in state of theart face detectors.As an additional comparison with prior work, we ran ourdetectors on the CMU/MIT dataset which contains 130 im-ages and 511 faces. Table 1 compares the results with previ-ously reported work using the output of Gabor detectors. Itshows that our Gabor based detector works significantly bet-ter than the Gabor detector proposed by Huang et al. [4], withan increased number of true positives and reduced number offalse positives. In addition, our detectors show a decreasednumber of false positives at a similar rate of true positivescompared to the Viola-Jones detector.The key advantage of our proposed discretization is itscomputational simplicity, while maintaining performancecomparable to that of VQ. In the implementation of our realtime face detector, we used a computer equipped with i5 2.67GHz CPU with an NVidia GTX-465 graphics card. For a640 × 480 image, there are 18 scales (pyramid levels), eachcomputed by resizing the input image with texture mappingon the graphics card. Computations to filter each level ofthe pyramid with the four Gabor filters using the algorithmin [10] and to compute the VQ and radial phase encodingare performed on the graphics card. The detector has twomain stages: building the pyramid and scanning the pyramid,as well as a clustering stage to merge multiple detections.The computational time required for each stage is shown inTable 2. The whole process of the radial-phase detector takes59.3 ms, corresponding to 16.8 fps. The efficiency of theVQ depends on the number of centers. On our system, the512 vector VQ detector takes 450.5ms and the 64 vector VQdetector takes 107.2ms. Both are significantly slower thanthe radial phase encoding. The main difference is in the timerequired to build the pyramid, since the VQ process requirescomparison with all of the vectors in the codebook. Here, thespeedup over VQ ranges from a factor of 3.5 for 64 vectors toover 20 for 512 vectors.4. CONCLUSION AND FUTURE WORKIn this work, we proposed an efficient discrete encoding oflocally normalized Gabor filter outputs, which outperformsstate of the art detectors on the FDDB dataset, and other Ga-bor detector based on continuous Gabor outputs. This encod-ing only requires several pixel-wise comparison operations onthe Gabor output. By implementing this method on a GPU,we achieved a real-time face detector with 16.8 fps.Gabor filters are widely used as image processing stepsfor extracting visual cues other than orientation. For exam-ple, disparity and motion cues can be calculated based onthe outputs of Gabor filter using the disparity and motion en-ergy models. Our work demonstrates that Gabor filter outputscan be effectively incorporated into face detectors. Our fu-ture work will seek to use these ideas to incorporate disparityand motion cues efficiently into face detectors. For example,these cues can be used to direct the scanning window insteadof scanning every pixel in the pyramid. This could lead tosignificant gains, since 65% of the computation is now spentfor scanning.5. REFERENCES[1] C. Küblbeck and A. Ernst, “Face detection and trackingin video sequences using the modifiedcensus transformation,”Image and Vision Computing, vol. 24, no. 6, pp. 564–572,2006.Table 1. Comparison of the detector performance on CMU-MIT setTrue positives False positivesVQ, 512 centers 444 6radial phase 444 17MCT baseline 430 7Gabor detector in [4] 421 57Viola-Jones in [2] 448 31Table 2. Comparison on the computational efficiency in (ms)Build ScanPyramid Pyramid ClusteringVQ, 512 centers 414.5 36.0 0.003VQ, 64 centers 71.7 35.5 0.003radial phase 21.0 38.3 0.027[2] P. Viola and M.J. Jones, “Robust real-time face detection,”International Journal of Computer Vision, vol. 57, no. 2, pp.137–154, 2004.[3] L. Xiaohua, K.M. Lam, S. Lansun, and Z. Jiliu, “Face detectionusing simplified Gabor features and hierarchical regions in acascade of classifiers,” Pattern Recognition Letters, vol. 30,no. 8, pp. 717–728, 2009.[4] L.L. Huang, A. Shimizu, and H. Kobatake, “Robust face detec-tion using Gabor filter features,” Pattern Recognition Letters,vol. 26, no. 11, pp. 1641–1649, 2005.[5] B. Kwolek, “Face Detection Using Convolutional Neural Net-works and Gabor Filters,” Artificial Neural Networks: Biolog-ical Inspirations–ICANN 2005, pp. 551–556, 2005.[6] J. Chen, S. Shan, P. Yang, S. Yan, X. Chen, and W. Gao, “Novelface detection method based on gabor features,” Advances inBiometric Person Authentication, pp. 49–94, 2005.[7] J. Chen, R. Wang, S. Yan, S. Shan, X. Chen, and W. Gao, “En-hancing human face detection by resampling examples throughmanifolds,” Systems, Man and Cybernetics, Part A: Systemsand Humans, IEEE Transactions on, vol. 37, no. 6, pp. 1017–1028, 2007.[8] V. Jain and E. Learned-Miller, “FDDB: A benchmark for facedetection in unconstrained settings,” Tech. Rep., Technical re-port, University of Massachusetts, Amherst, 2010. 111.[9] K. Mikolajczyk, C. Schmid, and A. Zisserman, “Human detec-tion based on a probabilistic assembly of robust part detectors,”Computer Vision-ECCV 2004, pp. 69–82, 2004.[10] X.X. Wang and B.E. Shi, “GPU implemention of fast Gaborfilters,” in Circuits and Systems (ISCAS), Proceedings of 2010IEEE International Symposium on. IEEE, 2010, pp. 373–376.

Effective Discretization of Gabor Features for Real-Time Face Detection

https://publikationen.bibliothek.kit.edu/1000026338/2020977

Effective Discretization of Gabor Features for Real-Time Face Detection

Abstract

Similar works

Full text

Available Versions

Crossref

KITopen