Color-segmentation is very sensitive to changes in the intensity of light.
Many algorithms do not tolerate variations in color hue which correspond, in
fact, to the same object. Learning Vector Quantization (LVQ) networks learn to
recognize groups of similar input vectors in such a way that neurons
physically near to each other in the neuron layer respond to similar input
vectors. Learning is supervised, the inputs vectors into target classes are
chosen by the user. In this work a new algorithm based on LVQ is presented. It
involves neural networks that operate directly on the image pixels with a
decision function. This algorithm has been applied to spotting and tracking
human faces, and shows more robustness than other algorithms for the same
task

Cuevas, Erik V

Rojas, Raúl

Zaldivar, Daniel

Institutional Repository of the Freie Universität Berlin

Competitive Neural Networks Applied to FaceLocalizationTechnical Report B-13-03Erik V. Cuevas1,2, Daniel Zald´ıvar1,2, Rau´l Rojas11 Freie Universita¨t Berlin, Institut fu¨r InformatikTakustr. 9, D-14195 Berlin, Germany2 Universidad de GuadalajaraAv. Revolucio´n No. 1500, C.P. 44430, Guadalajara, Jal., Me´xico{cuevas, zaldivar, rojas}@inf.fu-berlin.deJune 8, 2004AbstractColor-segmentation is very sensitive to changes in the intensity of light. Manyalgorithms do not tolerate variations in color hue which correspond, in fact, to thesame object. Learning Vector Quantization (LVQ) networks learn to recognize groupsof similar input vectors in such a way that neurons physically near to each other inthe neuron layer respond to similar input vectors. Learning is supervised, the inputsvectors into target classes are chosen by the user. In this work a new algorithm based onLVQ is presented. It involves neural networks that operate directly on the image pixelswith a decision function. This algorithm has been applied to spotting and trackinghuman faces, and shows more robustness than other algorithms for the same task.1 IntroductionIn computer vision applications based on color-segmentation there is a common problem:the sensitivity to changes in the intensity of light. This is a result of the inability of somealgorithms to tolerate variations presented in the color hue which correspond, in fact, to thesame object. In this work a new algorithm based on LVQ neural networks is presented. Itinvolves neural networks that operate directly on the image pixels with a decision function.1Figure 1: Architecture of a competitive networkThis algorithm has been applied to spotting and tracking human faces, and shows morerobustness than other algorithms for the same task.[1,2]. This work is organized in thefollowing way: in Section 2 an introductory study of competitive neural networks and theirmain characteristics are presented, in Section 3 some important details of LVQ networks areanalyzed, in Section 4 the architecture and characteristics of the proposed color-segmentationsystem are explained, in Section 5 the implementation of the algorithm is exposed, in Section6 the Neural Network (NN) creation and training in Matlab are explained while in Section7 the obtained results are reported.2 Competitive NetworksCompetitive Networks [3] learn to classify input vectors according to how they are groupedin the input space. They diﬀer from another networks in that neighboring neurons learn torecognize neighboring sections of the input space. Thus, competitive layers learn both thedistributions and topology of the input vectors which they are trained on.The architecture for a competitive network is shown in Fig. 1. The |Ndist| box in theﬁgure accepts the input vector p and the input weight matrix IW and produces a vectorhaving S elements. The elements are the negative of the distances between the input vectorp and the vector IW. The net value v of the competitive layer is computed by ﬁnding thenegative distance between input vector p and the weight vector IW and then adding thebiases b. If, all biases are zero, the maximum net input that a neuron can have is 0. Thisoccurs when the input vector p equals the neuron’s weight vector.The competitive transfer function C accepts a net value v and returns neurons outputs2Figure 2: Left, one dimensional neighborhood of radius d = 1.Right, neighborhood of radiusd = 2of 0 for all neurons except for the winner, the neuron associated with the most positiveelement of the input v. Thus, the winner’s output is 1.The weights of the winning neuron are adjusted with the Kohonen learning rule. Suppos-ing that the ith neuron wins, the elements of the ith row of the input weight matrix and allneurons within a certain neighborhood Ni(d) of the winning neuron are adjusted as shownthe Eq.1.iIW1,1(q) =i IW1,1(q − 1) + α(p(q)−i IW1,1(q − 1)) (1)Here α is the learning rate and Ni(d) contains the indices for all of the neurons thatlie within a radius d of the ith winning neuron. Thus, when a vector p is presented, theweights of the winning neuron and its closest neighbors move toward p. Consequently, aftermany presentations, neighboring neurons will have learned vectors similar to each others.The winning neuron’s weights are altered proportional to the learning rate. The weights ofneurons in its neighborhood are altered proportional to half the learning rate. In this work,the learning rate and the neighborhood distance (used to determine which neurons are inthe winning neuron’s neighborhood) are not altered during training.To illustrate the concept of neighborhoods, consider the Fig. 2. At left is shown a one di-mensional neighborhood of radius d = 1 around neuron 15, at right is shown a neighborhoodof radius d = 2.These neighborhoods could be written as:N15(1) = (14, 15, 16), N15(2) = (13, 14, 15, 16, 17)3 Learning Vector Quantization NetworksAn LVQ network [4] has ﬁrst, a competitive layer and second, a linear layer. The competitivelayer learns to classify input vectors like the networks of the last section. The linear layertransforms the competitive layer’s classes into target classiﬁcations deﬁned by the user. Werefer to the classes learned by the competitive layer as subclasses and the classes of thelinear layer as target classes. Both the competitive and linear layers have one neuron perclass. Thus, the competitive layer can learn S1 classes. These, in turn, are combined by thelinear layer to form S2 target classes. The LVQ network architecture is shown in the Fig. 3.3Figure 3: Schematic representation of the LVQ netFigure 4: Architecture of the color segmentation systemSummarizing, the LVQ network allows to order classes learned by the competitive networkin a ﬁnal vector more appropriate for the color-segmentation.4 Architecture of the color segmentation SystemThe core of the algorithm is a LVQ network whose inputs are connected directly with thepixel-vector of the image I (in RGB format) and its outputs are connected directly to thedecision function fd, which produces an output of 1 or 0 depending of if the color correspondsto the object to be segmented forming in this way a new vector image I’. The Fig. 4 showsa scheme of the segmentator.Considering that the LVQ net is conﬁgured with N output neurons, then it would bepossible train the competitive net to learn the conﬁguration space and the color pixels4Figure 5: Decision function modeltopology, described as the vector p with elements pR , pG and pB coming from the image.For the supervised training of the linear net, we suppose that the image I contains an ObjectO to be segmented, being pO a pixel corresponding to the object, we train the linear networkin such a way that the class S2 (of this pixel) corresponds to the assigned (in a supervisedway) for the neuron N/2 of the linear network.The idea is that the color to be segmented is located halfway of the network, while thesimilar colors are located in the neighboring neurons. In such a way that if exists a vector p1that belongs to the object but for the illumination conditions and noise could be classiﬁedin the neighborhood of the neuron N/2.The classiﬁcation achieved by the LVQ network gives a vector of elements categorized byS2 of N elements corresponding to the N classes. Each element of the S2 vector could have2 possible values, 1 or 0 and only an element from each vector could be 1, while the otherelements will be 0. Then for the object to be segmented, the activation of the neurons isconcentrated in the middle of the vector, that is the neurons nearest to the N/2 will havea bigger possibility to be activated than the other ones corresponding to the color to besegment.Considering the previous problem is necessary to deﬁne a function fd able to deﬁneneurons density which will be taken to consider if a pixel corresponds or not to the colorto be segmented, this function will be called in this work decision function. It is possibleto formulate many functions which could solve the decision problem satisfactorily, howeverthe Gaussian function has been chosen by its well-known properties, although it is possibleto use other, including non-simmetrical distributions. The Fig. 5 shows graphically thefunction used. The equation 2 shows mathematically this function where g is the numberof the activated neuron, μ is N/2 and σ is the standard deviation. Therefore, fd has only a5calibration parameter represented by σ which determines the generalization capacity of thecomplete system. Thus, if the value of σ is chosen small enough, the segmentation capacitywill be more selective than in the case of a bigger σ. For the ﬁnal result the decision functionwas evaluated with a threshold of 0.7.fd(g) =1√2πσexp(− (g − μ)22σ2)(2)5 ImplementationThe implementation is divided in two parts, the net training and the segmentator structure.In the ﬁrst part, a net of 30 neurons is chosen for the competitive layer and also for thelinear layer, then a frame of the image is taken containing the object whose color will besegmented, then a pixel corresponding to the color of this object is chosen and the trainingof the LVQ net is made it, allowing that the pixel-class corresponds to the 15th net class(N/2). During the training it is used a learning rate α of 0.1 and a distance radius Ni(d) of3. This conditions make sure that the winner weights are aﬀected in a reason of 0.1 whilethree neurons of its neighbors (in both ways) are aﬀected in a reason of 0.05. The processof training was made using the neural networks toolbox of Matlab.In the second part(the segmentator implementation), a decision function with parametersμ =15 and σ =3 was added to the net (above) generated by the training. The complete systemwas coded in C++ and tested on a PCx86 at 900 MHz with 128 MBytes RAM, operatingin real time on an image of 352x288 pixels surrendered by an USB-Webcam.To inspect the segmentator robustness, it was tested on face localization. The segmen-tator gives an image with a value of 1 in those points that belong to the segmented objectwhile in other cases this is 0. This is caused as a result that the object to be segmentedin this case, will have an appearance of a white ﬂeck. Thus, to ﬁnd the object position itwill be necessary to calculate the object centroid (5) through the zero moment (3) and ﬁrstdegree moment order (4).M00 =∑x∑yI(x, y), (3)M10 =∑x∑yxI(x, y),M01 =∑x∑yyI(x, y), (4)xc =M10M00, yc =M01M00, (5)where M00 represents the zero degree moment while of M10 and M01 means the ﬁrstdegree moments of x and y respectively, while xc and yc represent the center coordinates.The Fig. 6 shows this process.6Figure 6: Center coordinates xc and yc.6 Neuronal Network Creation and Training in Matlab6.1 Creating a NN in MatlabCompetitive layers and self organizing maps can be created with the Matlab commandsnewc and newsom, respectively. Also is possible specify diﬀerent topologies for the originalneuron locations with the Matlab functions gridtop, hextop or randtop.In the neural networks toolbox from Matlab, there are four distinct ways to calculatedistances from a particular neuron to its neighbors. Each calculation method is implementedwith a special function.In this work was used the function dist. It calculates the Euclidian distance from thehome neuron to any other neuron.Suppose that we want to create a network having input vectors with two elements thatfall in the range 0 to 2 and 0 to 1 respectively. Further suppose that we want to have 10neurons in a linear manner 1-by-10 network. The code to obtain this network is:net=newsom([0 2; 0 1],[1,10]);6.2 Training a NN in MatlabLearning, in a self-organizing map, occurs for one vector at the time, independent of whetherthe network is trained directly (with the Matlab function trainr) or wether it is trainedadaptively (with the Matlab function trans). In either case, learnsom is the self-organizingmap, weight learning function in Matlab.First, the neuron identiﬁes the winning neuron, the weights of the winning neuron andthen the neighboring neurons are moved closer to the input vector at each learning stepusing the self-organizing learning map (Matlab function learnsom). The parameters can tobe modiﬁed with the valuesLP.order_lr.7LP.order_steps.LP.tune_nd.Learning rate, Nummer of steps and neighborhood distance, respectively.Thus, we can train the network for 100 epochs, a learning rate of 0.5 and neighborhooddistance of 1. The code is:net.trainParam.epochs=100;net.trainParam.order_lr=0.05;net.trainParam.tune_nd=1;net=train(net,P);where P is the matrix with the training patterns.6.3 Creating and training the Linear LayerLinear layers can be created with the Matlab command newlin. For example, we can createa network with two elements that be in the range −1 to 1 and one output, like is show inthe followings commands:net=newlin([-1 1; -1 1], 1);Linear networks can be trained to perform linear classiﬁcation with the Matlab functiontrain. This function applies to each vector a set of input vectors and calculates the networkweights and bias increments due each of the inputs, according to learnp Matlab function.6.4 Auxiliary FunctionsFor the image capture the V FM utility was used, this Matlab-tool captures images(in RGBformat) in a m ∗ n ∗ 3 matrix(been mxn the dimensions of the image and the index 3 thecorresponding index for R,G and B). This data matrix is represented in a integer formatand it should be converted to a ﬂoat point matrix(double). An example code for an imagecapture is shown.vfm Im=vfm (’grab’,1);the captured image will be store in Im. Im should be transform for the training in a datavector of mxn type (R,G and B color channels) thats is performance by the next commands:R=Im(:,:,1);V=Im(:,:,2);A=Im(:,:,3);[m,n]=size(R);index=0;8for a1=1:mfor an=1:ndata=R(a1,an);data1=V(a1,an);data2=A(a1,an);index=index+1;VR(index)=data;VV(index)=data1;VA(index)=data2;endendVP=[VR;VV;VA];VP=double(VP);Thus, from VP (the training vector) is possible to perform the training. The followingfunctions sequens make that:net=newsom([0 255;0 255;0 255],30);net.trainParam.epochs=500;net.trainParam.order_lr=0.1;net.trainParam.tune_nd=3;net=train(net,VP);Ones, that the competitive neuronal network has been trained than it is trivial to ﬁnda point and their correct identiﬁcation in the data vector and then make able the super-vised training of the lineal network(using the Matlab function learnp). During the training,practically any parameters combination give good results.It is possible that the data obtained by the neuronal network could be interpreted as animage. To prevent this, a function was created, this function allows to modify the vectorformat to a mxn image format, the following commands show that:index=0;for a1=1:mfor an=1:nindex=index+1;data=Image(index);MR(a1,an)=data;endendwhere Image is the data vector obtained by the neural network.92040608010012014016020406080100120140−1−0.500.51(a)(b)Figure 7: (a) Classiﬁcation performed by a LVQ network with 10 neurons. (b) Final imageof the segmentator used for face tracking7 Conclusions.In this section the segmentator results obtained in 3 diﬀerent parts are presented.1. In the learning of the LVQ network was proved the net classiﬁcation robustness and itscapacity to organize topologically, which shows excellent results, being able to performthe pixel classiﬁcation of the image even in critical cases with very few neurons. TheFig. 7(a) shows the obtained classiﬁcation results from an image, using a LVQ networkwith only 10 neurons.2. The system was tested to track a human head showing very good results. The systemperformance proposed in this work compared with some other algorithms [1,2] wasgood with the advantage that is computationally more eﬃcient, being better for appli-cations where it is required, besides of the object localization, also to perform complex10calculations, for example the dynamics of a robot structure (that could supports thevision system). The Fig. 7(b). shows a tracked image in the ﬁnal application.3. The inﬂuence of the σ parameter in the decision function was demonstrate, and for thiscase values of σ in the interval of 3 at 5 gave good results. It can also be said that therobustness of the system can improve if it were possible to implement an algorithm toadapt the σ parameter. In the Fig. 8(a)-(b) shows the results obtained with diﬀerentvalues of σ.(a) (b)Figure 8: Results considering (a) σ = 4 and (b) σ = 1.References[1] Jang,G.J., Kweon,I.O.: Robust Real-time Face Tracking Using Adaptive Color Model.International Conference on Robotics and Automation,(2001)[2] Nummiaro, K., Koller-Meier, E., Van-Cool, L.: A color-based Particle Filter. Proceedingsof the 1st International Workshop on Generative-Model-Based Vision,(2002)[3] Kohonen, T.: Self-Organization and Asociative Memory. 2nd. edn., Springer-Verlag,Berlin,(1987)[4] Kohonen, T., Self-Organizing Maps, 2nd. edn., Berlin, Springer-Verlag,Berlin,(1997)11

Competitive neural networks applied to face localization

https://refubium.fu-berlin.de/bitstream/fub188/19518/1/2003_13.pdf

Competitive neural networks applied to face localization

Abstract

Similar works

Full text

Available Versions

Institutional Repository of the Freie Universität Berlin