We propose a new technique for a faster computation of the activities of the hidden layer units. This has been demonstrated on face detection examples

Ben-Yacoub, Souheil

Fasel, B.

Luettin, Juergen

Infoscience - École polytechnique fédérale de Lausanne

Fast Face Detection using MLP and FFTS  BenYacoub B  Fasel and J  LuttinIDIAPCP   MartignySwitzerlandemail  sbyidiapchAbstractNeural networks have shown their reliabilityand robustness for the face detection task  Rowley Baluja and Kanade  However thetime consuming process needed by neural networkshas prevented them from being a practical toolWe propose a new technique that signicantlyspeeds up the time needed by a trained network MLP in our case to detect a face in a largeimage We reformulate neural activities in thehidden layer of the MLP in terms of lter convolution enabling the use of the Fourier transformfor an e	cient computation of the neural activities The method was applied to face detection instill images as well as on live video sequences  IntroductionFace detection is a fundamental step before therecognition or identi cation procedure Its reliability and timeresponse have a major inuence onthe performance and usability of a face recognition systemThe large variability of human faces causes major diculties in the design of a model that canencompasses all possible faces  Appearancebased approaches as well as learningbased approaches seem to be best suited for this task Ineither case a set of representative faces is necessary to  nd an implicit modelEigenfaces 	 were used to model the distribution of faces in some large input space typically the input space is Rnwhere n is the sizeof the image The main assumption of this approach is that the set of faces are located in asubspace that can be approximated using theKLtransform through a training set The faceness of an input image is determined by itsdistance to the face subspaceA similar approach was proposed by Sung andPoggio  where the subspace was approximated by  gaussian distributions The distancesbetween a given input image and the  subspaces generated a vector that was used by aperceptron to separate the face space from thenonface spaceThe use of neuralnetworks appears to be oneof the most promising methods for face detection		 However the time consuming processing  needed by the neural networks has preventedthem from being a practical toolA neural network based face detector was proposed in  and has shown good results A feedforward neural network was designed to detectfaces using a   input window The neuralnetwork architecture was optimally designed withreceptive  elds to specialize a set of neurons todetect eyes mouth and nose The negative examples ie nonface images for the trainingset were generated using a bootstrap techniqueThe system has demonstrated excellent detectionrates but it suers from time consuming computations yielding slow responses Most of thecomputation time is spent in exploring all thepossible subimages Some strategies were usedto reduce the time complexity but with the drawback of a lower system performanceWe present an approach that speedsup theprocessing time by considering a MLP MultiLayer Perceptron as a bank of  lters and by reformulating the processing steps in terms of con                                                                                                                                                                                                                             Input Layer Hidden Layer Output LayerW1 W2Input ImageSliding windowFigure 	 MLP architecture for object detectionvolutions Performing the convolution in the frequency domain is less time consuming The characteristic of this approach is that it considerablyreduces the computation time while maintainingidentical performance This method enables afast computation of the MLP neuron activitiesbut does not reduce the recognition performanceof a MLP based detector The method has beenvalidated for face detection in still images as wellas in live video sequences The FFTMulti Layer PerceptronWe consider a layer feedforward neural network or MLP see Figure 	 trained with theclassical backpropagation algorithm The inputlayer is a vector and the output layer is a singleneuron for the sake of simplicity  Let I be the input layer vector H the hidden layer vector and Othe output We consider n n pixel input imagestransformed into a columnvector to feed the input layer constituted by nunits or neurons Thehidden layer has m neurons and the output layeris a single neuron which is triggered to 	 if thelearned pattern is present and  otherwiseThis architecture can be used to detect dierent objects Changing the learning set generatesa new object detector The modular property ofthis architecture makes it very exible and handyThe activity of a particular neuron i in the hidden Extension to an output with multiple neurons isstraightforward layer H can be written ashi gnXj W ijIj b i 	Where W iis the set of weights of neuron i andb i is the threshold Similarly the output layeractivity isO  gmXj Wjhj bWhere Wjare the output weights and btheoutput thresholdThe training algorithm is based on the backpropagation of the error which is now a wellknown technique  An example is picked fromthe training set the output is computed Theerror is computed as the dierence between theactual and the desired output It is minimized bybackpropagating it and by adjusting the weightsDuring the recognition step of the classicalMLP a subimage of size n n is extracted fromthe test image of size N N  and fed to the neuralnetwork This operation must be iterated on allpossible dierent subimages of the input imageThis is the major drawback in the use of neuralnetworks for object recognition Typically for anN N test image from which all n n subimagesare extracted to compute the activities ofm neurons in the hidden layer ONnm computationsteps are requiredTo reduce the computational burden we propose to eciently calculate the activity of hiddenunits using the Fourier transform The activityof hidden neuron i with weight matrix i overthe whole image I can be formulated as a crosscorrelation operationHi gI  iB  where B k  l  b i k  l  	n The matrix Hiis the activity matrix of the hidden uniti From equation  we can state that Hij  kis the activity or output of the hidden unit iThe crosscorrelation operation is represented by  when the observation window is located at position j  k in the input image I The  nal outputactivity matrix of the neural network can then beexpressed as a linear combination of the hiddenunits activityO  gmXi WiHi b Here again Oj  k is the output of the neuralnetwork when the observation window is locatedat j  k in the input image IIn equation  the activity is expressed interms of a crosscorrelation between a bank of lters ii     mand the input image I Theadvantage in this reformulation is that crosscorrelation can be performed eciently in frequency domain using the following relationI    F FI  FThe D fast Fourier transform D FFT ofa N   N test image I requires ONLogNcomputation steps The D FT of the  ltersii     mcan be computed oline since theyare constant parameters of the network independent from the image A D FT of the test imagehas to be computed therefore the total numberof FT to compute is m  	 yielding a total ofOm  	NLogN computation steps Thespeedup factor ismnm LogN In our experiments we used  hidden units ie m and   pixels subimages ie n The curvegiving the speedup factor with respect to imagesize is shown in Figure Figure  Speedup curve with respect to N Image size N  N Face detection using MLPWe applied the reformulation of MLP in termsof Fourier transform to the face detection taskThe training images are     pixels largeand represent human faces and nonface patternsThe examples were taken from the MVTS database which contains frontal views of  different persons The negative examples were generated from images without faces mostly textureimages and also by using a bootstrap procedure The MLP was trained on 	 face examplesand  nonface examplesThe faces in the training set have slightly different scales There are also some shifted faces	 to  pixels o center Face images often leadto multiple detections see Figure  a whereasnonface images mainly lead to single detectionsin the case of false detection This informationwas used to remove a large number of false detectionsa bFigure  Example of multiple detections  Greylevel normalizationIn order to achieve a detection robust towardillumination changes we transform the images ofthe learning set into zeromean normalized vectors If fXi  i  	Lgis the set of imagesthe following holds for the transformed imagesXii    LkXik 	 and meanXi    i  	LIn the recognition step the data must be also normalized and have zeromean A problem arisesL is the size of the training sethere the normalization and the centering	if applied on the whole image will not be guaranteedlocally in the subwindowsThe normalization and centering must be applied on the extracted subimage and not on thewhole input image and this is not straightforwardsince Fourierbased convolution is a global processing The reformulation of the local normalization and centering of the data in the FT framework in 	 shows that convolving a centered imageI with a  lter  is equivalent to convolvingthe noncentered image I with the centered  lter The centered  lter can be computed olineand thus does not generate extra computation Multiscale face detectionMultiscale face detection is achieved by building a set of multiresolution images The facedetector is then applied at each resolution andthe  nal output is a combination of the dierentlevelsNote that the FT of the lower resolution imagesis not computed due to the scaling property of theFT If fx  y is the original image and F u  v itsFourier transform and gx  y is the subsampledby a factor  in each direction image then wehave the following propertyFT gx  y  Gu  v 	F u v Figure  a displays the detections at all levelsof resolution the level of resolution is given bythe size of the square The  nal output is givenin Figure  b It takes into account the numberof times a region was detected as face for moredetails see a bFigure  Multiscale face detectionCentered image stands for zeromean image Experimental Results Still imagesTesting was performed on realistic and complex images extracted from the CMU databaseExperiments were performed on test set A images from CMU database The achieved detection rate using a single MLP  neurons inhidden layer is in the range  to  of facesdetected see Figure  ai The system proposed by Rowley achieves a detection rate rangingfrom  to  depending on the used heuristic Our approach generates more false alarmsthat could be reduced by combining multiple networks and by increasing the size of the trainingdatabase The higher performance obtained byRowley  is likely to be due to a much largertraining set and due to a combination of multiplemodular networks Video SequencesThis algorithm was also applied to face detection in live video sequences using a camera anda videoboard The algorithm runs on an UltraSparc  with an Osprey 	 video boardThe processed frames are 	  	 pixels largeThe face detector was tested in an oce environment with dierent illumination conditions anddierent backgrounds Each frame is processedin about  seconds without using any temporalor spatial information the face detection is performed over the whole frame at each step We expect to reduce the processing time by introducingtemporal knowledge The system showed goodperformance even if multiple faces were presentin the frame It was tested with dierent peoplevisitors from outside the institute SomeMPEGmovies of face detections are available on theWWW ConclusionWe proposed a simple and exible MLP architecture for fast object detection face detectionfor example The main contribution of the paper was to describe a method which reduces dramatically the computation time of a MLP basedhttpwww cs cmu eduharhttpwww idiap chvisionfacerecognition htmldetector without altering the performances Thereformulation of MLP in terms of  lter convolutions enabled us to speedup signi cantly the processing time Classical approaches have to preprocess the data during runtime for normalization purpose in our case no preprocessing isneeded since the normalization is incorporateddirectly in the weights of the network ie coecients of the  lters The same algorithm can beused to detect other features eyes mouth noseetc separately by changing the learning setAcknowledgmentThis work was supported by the Swiss FederalOce for Education and Science and the ACTSMVTS project The author used the test database provided by CMU H Rowley S Baluja andT Kanade and AI Lab MIT K Sung and TPoggioReferences	 Souheil BenYacoub Fast object detectionusing MLP and FFT IDIAPRR 		 IDIAP	 R Chellappa CL Wilson and CS BarnesHuman and machine recognition of faces Asurvey Technical Report CARTR	 University of Maryland USA 	 M Collobert R Feraud G Le Tourneurand O Bernier Listen A system for locating and tracking individual speakers InndInternational Conference on AutomaticFace and Gesture Recognition pages  Oct 	 B Fasel Fast multiscale face detectionIDIAPCOM  IDIAP 	 S Pigeon The mvts multimodal face database release 	 CEC ACTSMVTSDeliveralble ACUCLWPDSP	 httpwwwteleuclacbe MJT Reinders RWC Koch and JJ Gerbrands Locating facial features in image sequences using neural networks In ndInternational Conference on Automatic Face andGesture Recognition pages  Oct	 HA Rowley S Baluja and T Kanade Human face detection in visual scenes Technical Report CMUCS	R CarnegieMellon University 	 DE Rumelhart and JL McClelland Parallel Distributed Processing Explorations inthe Microstructure of Cognition MIT Press	 K K Sung and T Poggio Examplebasedlearning for viewbased human face detecttion Technical Report AIMemo 		 Massachusetts Inst of Technology CambridgeMA USA 		 M A Turk and A P Pentland Eigenfacesfor recognition Journal of Cognitive Neuroscience  		 				 D Valentin H Abdi AJ Otoole and GWCottrell Connectionist models of face processing A survey Pattern Recognition		 	a b cd e fg h ij k lm n op q rFigure  Face detection results on static images and video sequences

Fast Face Detection using MLP and FFT

http://infoscience.epfl.ch/record/82563/files/avbpa_face.pdf

Fast Face Detection using MLP and FFT

Abstract

Similar works

Full text

Available Versions

Infoscience - École polytechnique fédérale de Lausanne