This paper presents a mechanism for analysing the deformable shape of an object as it moves across the visual field. An object’s outline is detected using active contour models, and is then re-represented as shape, location and rotation invariant axis crossover vectors. These vectors are used as input for a feedforward backpropagation neural network, which provides a confidence value determining how ‘human’ the network considers the given shape to be. The network was trained using simulated human shapes as well as simulated non-human shapes, including dogs, horses and inanimate objects. The network was then tested on unseen objects of these classes, as well as on an unseen object class. Analysis of the network’s confidence values for a given animated object identifies small, individual variations between different objects of the same class, and large variations between object classes. Confidence values for a given object are periodic and parallel the paces being taken by the object

Adams, R.G.

Davey, N.

George, S.

Tabb, Ken

English

University of Hertfordshire Research Archive

The Analysis of Animate Object Motion using Neural Networks andSnakesKen Tabb, Neil Davey, Rod Adams & Stella Georgee-mail {K.J.Tabb, N.Davey, R.G.Adams, S.J.George}@herts.ac.ukhttp://www.health.herts.ac.uk/ken/vision/Department of Computer Science,University of Hertfordshire,College Lane, Hatfield, Herts, UK. AL10 9ABAbstract: This paper presents a mechanism for analysing the deformable shape of anobject as it moves across the visual field. An object’s outline is detected using activecontour models, and is then re-represented as shape, location and rotation invariant axiscrossover vectors. These vectors are used as input for a feedforward backpropagation neuralnetwork, which provides a confidence value determining how ‘human’ the networkconsiders the given shape to be. The network was trained using simulated human shapesas well as simulated non-human shapes, including dogs, horses and inanimate objects.The network was then tested on unseen objects of these classes, as well as on an unseenobject class. Analysis of the network’s confidence values for a given animated objectidentifies small, individual variations between different objects of the same class, andlarge variations between object classes. Confidence values for a given object are periodicand parallel the paces being taken by the object.Keywords: Human, Motion Analysis, Shape, Snake, Active Contour, Neural Network,Axis Crossover Vector1 IntroductionThis paper outlines a mechanism for identifying and classifying the shape deformation and motionpattern of walking humans and other animate objects. The method combines active contour models(‘snakes’) [Kass, Witkin & Terzopoulos 1988], with a categorisation neural network [Tabb et al. 1999aand 1999b]. A snake is used to detect the outline of an object in an image, having been manuallyinitialised around the object by the user. Once the object’s outline has been detected, the resultantcontour is re-represented in a scale, location and rotation invariant vector. These vectors are then used totrain a feedforward backpropagation neural network to distinguish human shapes from non-humanshapes. Once trained, further unseen vectors are presented to the network to determine how accurate thenetwork is at classifying human and non-human shapes. Snakes are then used to track animate objectsin sequences of images, and the subsequent shape vectors are presented to the network to identifypatterns in a given object’s shape deformation. A discussion of these deformation patterns is given,along with possible uses of the technology.The techniques presented are part of a larger system designed to track moving pedestrians, aproblem that has been the subject of much research [Baumberg & Hogg 1994; Bowden, Mitchell &Sarhadi 1998; Galanta, Johnson & Hogg 1999]. We show that the periodic nature of human walking isclearly discernible from the deformation pattern, and that individual humans have a specific temporalpattern.2 Identifying and Representing Moving ObjectsIn order to analyse an animate object’s deformable shape during motion it is first necessary to obtainthe shape of that object in each of a series of images. Snakes are an established method (Blake & Isard1998) for identifying and tracking moving objects. Snakes are energy minimising splines whoseaccuracy at detecting an object is dependent upon the how suitably defined the snake’s energy functionhas been. A snake’s energy can contain different types of shape criteria, such as the goal to staycircular, as well as different image criteria, such as having lower energy associated with, and thus beingattracted to, edges or specific colours in the image. The combination of these criteria results in anenergy function tailored to a specific task.The user initialises a snake in an image or movie frame, and the snake relaxes around thetarget object until it finds a local or global minimum in the energy space. The snake, once relaxed in aminimum, can then be introduced into the next frame of a movie, and re-minimised, to track a movingobject autonomously (Figure 1).An active contour model based on Fast Snakes [Williams & Shah 1992] has been designed forthe task, and provides a reasonably accurate means of identifying and tracking an object’s shape duringmotion (Figure 2).Frame 1Frame 1 Frame 2Figure 1: Detecting and tracking objects using an active contour. [Left] The user initialises acontour around the target human. [Middle] Minimising the snake’s energy forces the snake torelax onto the human outline. [Right] Once relaxed, the snake is initialised in the next frameusing its relaxed position as a starting point. Its energy is then minimised again to relax itonto the human’s new position.0 15 30 45 60 75 90 105Figure 2: Human poses (grey) being tracked with an active contour (black). Numbers beneatheach pose denote the movie frame number.Due to an active contour’s scale, location and rotation dependence, and additionally because it is storedas pairs of (x,y) coordinates, the contours are not suitable to use as input patterns for neural networks.Instead, an active contour can be translated into a scale, location and rotation independent axis crossovervector [presented in Tabb et al. 1999a]. A number of axes are projected outwards from the contour’scentre (Figure 3), with the distance from the contour’s centre to the furthest contour edge along eachaxis being stored in a vector. This vector is then normalised by its largest element, making the vectorscale, location and rotation invariant.[196, 134, 63, 77, 72, 31, 49, 206, 21, 204, 45, 39, 68, 90, 103, 159]1   axis16   axis[0.95, 0.65, 0.31, 0.37, 0.35, 0.15, 0.24, 1.0, 0.1, 0.99, 0.22, 0.19, 0.33, 0.44, 0.5, 0.77]stthFigure 3: Converting an active contour into an axis crossover vector. Axes are projected atspecific angles from the contour’s centre to its edges. These axes’ distances are then stored in avector and normalised, making the vector scale, location and rotation invariant.3 Identifying Human Shapes with Neural NetworksA range of experiments have previously been performed to validate the axis crossover’s ability torepresent contours sufficiently for a neural network to be able to distinguish human from non-humanshapes [Tabb et al. 1999b]. This paper uses exclusively a network with 16 input units and 2 outputunits, where one output unit was trained to identify human shapes, and the other non-human shapes.All axis crossover vectors used contained 16 elements, where a given element maps onto a given inputunit in the neural network (Figure 4).Output Layer ...[0.95, 0.65, 0.31, 0.37, 0.35, ...]......Axis crossover vectorInput LayerHidden LayerFigure 4: Axis crossover vectors as input patterns for neural networks. A given vector elementmaps onto a given input unit in the neural network, thus the size of input layer and axiscrossover vector must equate.In order to produce clean and reasonably varied sets of training and test patterns, simulated shapes ofhumans and non-humans were used. The 3D modelling and animation package Poser [Metacreations1999] was used to generate all simulated human, equine, canine and velociraptor movement. Thetraining set contained 400 simulated human and 400 simulated non-human shapes. The non-humanshapes consisted of inanimate objects such as trees, cars and streetlights; and deformable animateobjects, namely shapes of dogs and horses. A breakdown of the training set can be seen in Figure 5. Allanimate movement consisted of still images or movies of the object (human, dog, horse orvelociraptor) walking from the left side of the image to the right. Each image or movie was different,either in terms of the physical build of the object, for example height and weight of object, or in termsof its walking habit, for example one human might swing their arms more than another. Generatingthis data using Poser allowed for much more variation in shapes and motions than could be achieved ina reasonable time frame using real images.Once trained, a scalar confidence value was obtained from the two output units which allows acrude measure of how ‘human’ the network considers a given vector to be. The confidence value issimply the difference of the two output unit values. Humans (400)Horses (100)Dogs (100)Inanimate Objects (200)Figure 5: Training data set for the neural network.Two experiments were performed in this study: a categorisation experiment to determine how accuratethe network was at classifying static shapes as human or non-human, and a tracking experiment toanalyse a given object’s deformable shape during motion.In the training set, the animate objects were divided into bipedal humans and quadrupedal dogsand horses. In order to explore the nature of the induced classification, we introduced another bipedalclass to act as ‘near-miss’ humans. The only other animated biped available in the software used was avelociraptor, a roughly human sized bipedal dinosaur.3.1 Categorisation experimentThe categorisation experiment involved presenting the trained network with a test set of bothhuman and non-human axis crossover vectors and measuring how successfully the network categorisedthem, based on the network’s confidence value for each shape. The test set for the categorisationexperiment contained 100 unseen human shapes, and 100 unseen non-human shapes (see Figure 6a).3.2 Tracking experimentThe tracking experiment involved tracking human and animate non-human objects using activecontours, and for each object, converting the sequence of relaxed contours into axis crossover vectors,then presenting each vector to the neural network serially. A confidence value versus time graph canthen be plotted, depicting how human the object was considered to be in each frame of its motion. Thetest set for the tracking experiment contained 30 unseen humans and 15 unseen non-human animateobjects (see Figure 6b). Each object was tracked for 2 consecutive paces.a) Categorisation test patterns Humans (100)Velociraptors (25)Horses (25)Dogs (25)Inanimate Objects (25)b) Tracking test patterns Humans (30)Velociraptors (5)Horses (5)Dogs (5)Figure 6: Test data set for the categorisation [left] and tracking [right] experiments. Noinanimate object shapes were used in the tracking experiment. All shapes used as test patternswere previously unseen by the neural network.4 Animate Object Motion Analysis using Neural Networks4.1 Categorisation experimentThe results for the categorisation experiment on unseen data can be seen in Figure 7. As hasbeen observed in previous experiments [Tabb et al. 1999a and 1999b], the network learns to classifyunseen data very successfully. When the novel near-miss velociraptor class is presented, the resultsshow that the network considers it to be more ‘human’ than ‘non-human’. Nevertheless it is notclassified with as great a level of confidence as the genuine humans. In other words the networkidentifies the ‘near-miss’ class as a near-miss. Human Velociraptor Horse Dog Inanimate0.00.20.40.60.81.01.2Output ValueObject classOutput Unit 1 Output Unit 2Figure 7: Analysis of neural network categorisation values for different types of unseenobjects. The network’s confidence value for a given shape can be obtained by subtracting thefirst output unit’s value from the second. Results shown reflect the mean values given for 100unseen human objects, 25 unseen velociraptors, 25 unseen horses, 25 unseen dogs and 25unseen inanimate objects. Also shown are the standard deviations for each object class.4.2 Tracking experimentThe results for the tracking experiment can be seen in Figure 8. Graphs are shown for 3 of the30 humans in the test set, namely those which the network categorised with most, average, and leastconfidence. Graphs are also shown for the velociraptor, dog and horse categorised with averageconfidence for their respective object class. The two consecutive paces tracked for a given object havebeen superimposed on each object’s graph.For each test object, the network’s confidence value varies with the object’s motion and iscyclical, with the frequency being once per pace. Furthermore, whilst a given object’s confidencepattern is not identical from one pace to another due to slight variations between paces, its confidencepatterns from pace to pace were more similar to each other than to those of other objects in the samespecies, identifying individual differences apparent in objects’ movement.These differences were even more marked when comparing objects from different species;Figure 9 shows the same objects’ confidence values being plotted against each other, with each object’stwo consecutive paces being plotted end to end. It is evident that the neural network considers somespecies to be more human than others, as was the case in the categorisation experiment. Furthermorebands can be drawn across the graph, marking off each species from each other.00.20.40.60.810 10 20 30 40 50 60Confidence ValueFrameBest Human00.20.40.60.810 10 20 30 40 50 60Confidence ValueFrameMidrange Human00.20.40.60.810 10 20 30 40 50 60Confidence ValueFrameWorst Human00.20.40.60.810 10 20 30 40 50 60Confidence ValueFrameVelociraptor-1-0.8-0.6-0.4-0.200 10 20 30 40 50 60Confidence ValueFrameDog-1-0.8-0.6-0.4-0.200 10 20 30 40 50 60Confidence ValueFrameHorse1st Pace 2nd PaceFigure 8: Motion analysis of animate objects. The three graphs on the left show best, typical,and worst human categorisation over time. The graphs on the right show the same informationfor a typical velociraptor, dog and horse. In each graph, 2 consecutive paces for theappropriate object have been plotted, showing more clearly each object’s repetitive yetdistinctive motion pattern. -1-0.8-0.6-0.4-0.200.20.40.60.810 20 40 60 80 100 120Confidence ValueFrameBest HumanMidrange HumanWorst HumanMidrange VelociraptorMidrange HorseMidrange DogFigure 9: Comparison of neural network confidence values during animate object motion. Eachobject has been tracked for two paces, where the second pace starts at frame 61.5 DiscussionIn this paper we have shown how snakes can be used to identify an object in an image, and then totrack a moving object. The resultant contours can be re-represented as scale, location and rotationinvariant vectors. A neural network was trained using these vectors and was then able to successfullyclassify a range of unseen objects. The confidence value generated by the trained neural network forsnapshots of a moving object form a periodic waveform individual to that object. A more detailedanalysis of the network’s confidence values for each given animated object class identifies small,individual variations between different objects of the same class, and large variations between objectclasses. One possible use of the object’s periodic waveform is to aid in identifying single paces of theobject, so that the object’s speed of motion can be gauged. Moreover, individual differences betweenobjects are apparent and could be used as identification tags.Other applications of the object’s periodic waveform might be to identify the object’s species,or to aid in predicting the object’s imminent motion by ‘fast-forwarding’ along its waveform.ReferencesBaumberg A. M. & Hogg D. C. [1994]. An efficient method for contour tracking using active shapemodels. University of Leeds School of Computer Studies Research Report Series, Report 94.11.Blake A. & Isard M. [1998]. Active Contours. Springer-Verlag.Bowden R., Mitchell T.A. & Sarhadi M. [1998]. Reconstructing 3D Pose and Motion from a SingleCamera View. In Proceedings of BMVA 1998.Galanta A., Johnson N. & Hogg D. [1999]. Learning Behaviour Models of Human Activities. InProceedings of BMVA 1999.Kass M., Witkin A. & Terzopoulos D. [1988]. Snakes: active contour models. In InternationalJournal of Computer Vision (1988), pp. 321-331.Metacreations [1999]. Poser 4 for Macintosh. http://www.metacreations.com/Tabb K., Davey N., George S. & Adams R. [1999a]. Detecting Partial Occlusion of Humans usingSnakes and Neural Networks. In Proceedings of the 5th International Conference on EngineeringApplications of Neural Networks, 13-15 September 1999, Warsaw, pp. 34-39.Tabb K., George S., Adams R. & Davey N. [1999b]. Human Shape Recognition from Snakes usingNeural Networks. In Proceedings of the 3rd International Conference on Computational Intelligenceand Multimedia Applications, 23-26 September 1999, Delhi, pp. 292-296.Tabb K., Davey N. & Adams R. [1999c]. Analysis of Human Motion using Snakes and NeuralNetworks. To be published in Proceedings of Articulated Motion & Deformable Objects 2000, Palmade Mallorca, September 9-10 2000. Submitted in December 1999.Williams D.J. & Shah M. [1992]. A Fast Algorithm for Active Contours and Curvature Estimation.In CVGIP - Image Understanding 55, pp.14-26.

A Fast Algorithm for Active Contours and Curvature Estimation.

Active Contours.

An efficient method for contour tracking using active shape models.

Analysis of Human Motion using Snakes and Neural Networks. To be published in

Human Shape Recognition from Snakes using Neural Networks.

Poser 4 for Macintosh. http://www.metacreations.com/

Reconstructing 3D Pose and Motion from a Single Camera View.

Snakes: active contour models.

The analysis of animate object motion using neural networks and snakes

http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.24.4134

The analysis of animate object motion using neural networks and snakes

Abstract

Similar works

Full text

Available Versions

University of Hertfordshire Research Archive