The intimate relationship between human walking and running lies within the skeleto-muscular structure. This is expressed as a mapping that can transform computer vision derived gait signatures from running to walking and vice versa, for purposes of deployment in gait as a biometric or for animation in computer graphics. The computer vision technique can extract leg motion by temporal template matching with a model defined by forced coupled oscillators as the basis. The (biometric) signature is derived from Fourier analysis of the variation in the motion of the thigh and lower leg. These signatures can be used for recognition by running or by walking. In fact, the mapping between these gait modes clusters better than the original signatures (of which running is the more potent) and can be used for recognition purposes alone, or to buttress both of the signatures. Moreover, the two signatures can be made invariant to gait mode by using the new mapping

Carter, John N.

Nixon, Mark S.

Yam, ChewYean

English

Southampton (e-Prints Soton)

  1) 2 sin( ) ( y y y y t A t S φ + ω =  On the Relationship of Human Walking and Running:  Automatic Person Identification by Gait   Abstract The intimate relationship between human walking and running lies within the skeleto-muscular structure. This is expressed as a mapping that can transform computer vision derived gait signatures from running to walking and vice versa, for purposes of deployment in gait as a biometric or for animation in computer graphics. The computer vision technique can extract leg motion by temporal template matching with a model defined by forced coupled oscillators as the basis. The (biometric) signature is derived from Fourier analysis of the variation in the motion of the thigh and lower leg. These signatures can be used for recognition by running or by walking. In fact, the mapping between these gait modes clusters better than the original signatures (of which running is the more potent) and can be used for recognition purposes alone, or to buttress both of the signatures. Moreover, the two signatures can be made invariant to gait mode by using the new mapping. 1. Introduction  There is much research in analysing human gait via computer vision, e.g. human motion (walking) tracking [1], gender [2] and age group [3] discrimination, and using walking gait as a biometric via techniques such as velocity moments [4], static body parameters [5], spatio-temporal patterns [6] and Fourier analysis of the thigh motion based on a simple pendulum model [7]. However, the potential of both walking and running gait [8,9] as cues for person identification has only been explored recently. And these model-based approaches include analysis of the thigh and lower leg motion. Nevertheless, the relationship between human walking and running (in computer vision) remains imperfectly understood. Due to the fact that these two gaits are derived from the same skeleto-muscular system, there must exist some correlation between them. Perhaps what makes it seem unfeasible is that human gait is not only a physiological property, but also a complex behavioural [10] characteristic. That is, we learn how to walk and run when growing up and individuals with similar physiological traits may have their own particular way of walking and running. An understanding of the relationship between human walking and running is essential not only for further improving the existing automatic person recognition system using gait, but also as a foundation for other studies e.g. biomechanics, robotics and computer graphics animation. To determine a relationship between these two biomechanically distinct gaits, we deployed the first analytical human gait model [9] which is invariant to walking and running, to automatically extract the lower limbs’ motion using computer vision. Then generic formulae are developed to describe the relationship of the two different gaits. This is the mapping which is based on the idea of phase modulation. The mapping is then used to create signatures that are invariant to walking and running, and enhance the original gait signature. The analysis here has been applied to the largest database of its kind, comprising 20 subjects who are walking and running.  2. Feature Extraction     The edge maps (Fig. 1a) obtained by applying the Sobel edge operator with a threshold on the horizontal component are used in an evidence gathering process to extract the lower limb's motion. The angles of rotation when walking and running are extracted automatically via a 2-pass evidence gathering technique with an analytical model [9] as the underlying template. This model includes the vertical motion of the hip (Eq. 1), the thigh rotation (Eq. 2) and the lower leg rotation (Eq. 3). Note that only the hip's vertical motion is included since the subjects are filmed walking and running on a motorised treadmill, and the resolution of the images used during the feature extraction is relatively low. Hence, the horizontal motion is insignificant enough to be ignored. The hip's relative vertical motion, Sy, is given as a function of time t by     (1) where Ay is the amplitude of the vertical oscillation and φ y is the phase shift. The frequency is twice the fundamental, ω y, since one gait cycle consists of two steps. The thigh rotation,  θ T, and the lower leg rotation, θ K, are derived from forced coupled oscillators joined in series where the upper and lower penduli model θ T and θ K, respectively, (see Fig. 1b) as described by    (2)  (3)where  K K T T m b m a ω = ω = , . A, B, C, D and E are ()() T T T T M at B at A + φ + + φ + = θ sin cos() () [() () () () ] K T T T b a mKM M Fat B Fat AFbt D Fbt C EKT + + φ + + φ +− + = θ−ω sin cossin cos2 22  2constants, F is a scaling factor, M is an offset, ω  is the fundamental frequency, m is the mass and φ  is the phase shift. A 2-pass evidence gathering technique first determines the best parameters of the model that best fits the whole sequence of θ T  and  θ K of a gait cycle by temporal template matching. Temporal template matching takes into account the whole sequence of images enabling motion analysis, i.e. capturing motion characteristics. Then the search is refined to focus on individual frames to determine the most accurate possible leg movement via local template matching. In other words, we measure the deviation from the individual's unique norm. Fig. 1c-d show the result of the feature extraction technique for running and walking for identical twins. The exceptional advantage of this model-based feature extraction technique is that it will only extract shapes that resemble and move like a pair of human legs. (a)                   θ T       mT                        θ K                             mK (b)  (c) (d) Fig. 1: (a) Single-edge map; (b) Pendulum model (c,d) Feature extraction results for running and walking. 3. Gait Signature The gait signature of a particular subject consists of the phase and the magnitude components of the Fourier description of the thigh and lower leg rotation measured from a gait cycle. Here, the significant features of periodic motion are captured. As a shift in the time domain will affect the phase in the frequency domain, the time domain signals are aligned to start at the same point, which is the minimum of the thigh rotation and the corresponding instance of time of the lower leg rotation. This ensures the validity of using the magnitude and the phase component when creating the gait signature for comparison. By using data between two successive minima, recognition becomes invariant to speed as data within a full gait cycle is used for recognition purposes. Although the magnitude spectra show some discriminatory ability [7], gait is not only characterised by exaggeration in the range of motion, but also involves the central pattern generator and musculature that together control the way the limbs move. As such, the magnitude components are multiplied by their respective phase components, to yield the phase-weighted magnitude (PWM),  PWM(ω ) =  () ()ω ω Θ ∠ • Θj j e e   (4) where  Θ (ejω ) is the Discrete Fourier Transform of a rotation, and •  denotes the multiplication of each element in the vector, thus increasing discriminatory capability. The zero-order term is ignored to eliminate the effect of any offsets, so the gait signature contains only the features of the pure motion of a gait cycle. The gait signature is formed from the lower order PWM components of both the thigh and the lower leg rotation due to their greater magnitude. The higher order harmonics are very small in magnitude and they are more likely to be dominated by noise.  4. The Relationship   There is a unique mapping between the walking and running signature for each subject, however, a generic mapping across the population does not exist, at least not by this approach. This prevents us from being able to deduce the relationship of an unknown walker given the signature for running based on other observations in the database, and vice versa. This is expected since an average human does not exist [11]. Even identical twins (in our database) who are identical by their visual appearance and physiological traits appear unique in the way they walk and run. This suggests that gait is not only a physiological characteristic, but also a behavioural trait. Fig. 2 shows the twins' gait signatures for walking and running where different symbols represent the two different subjects. (a) (b) Fig. 2: Signature for (a) running and (b) walking. x: 1st component of θ T ; y: 1st  component of θ K ; z: 2nd component of θ K. In the time domain, running can be considered as a phase modulated version of walking (and vice versa), that is, the range of movement made by the lower limbs and the manner in which the limbs swing when walking and running are different. If f(ω t) is the signal for walking and the phase modulating signal is Kejψ (t), then the modulated signal (for running) is () ) ( ) ( t t Kf t g ψ + ω = ω   (5) and likewise when transforming the mode of running to that of walking gait. In the frequency domain, the spectra of the signals for walking and running can be easily related by their phase difference and magnitude ratio of each harmonic and these lead to the mapping, T, for n   3nn n TR W MM M =] , [n n T T n M T φ =harmonics as [] n T T T ,... , 2 1 = T   (6) Each element in T consists of the phase difference n T φ  and the magnitude ratio n T M  as   (7) where n T φ is the difference between running, n R φ , and walking, n W φ  n n n W R T φ − φ = φ   (8)  andn T M  is the ratio of running, n R M , to walking, n W M    (9) This mapping process shows how these two different gait can be associated by the phase and the magnitude in the frequency domain. The mode of gait can be transformed easily by altering the phase and the magnitude of each harmonic according to its mapping. Likewise, the gait signature can be made invariant by transforming it from one form to another. Fig. 3 shows the phasor diagram of the mapping vector T1 (the first harmonic) of θ T  and  θ K for 6 subjects. As shown, the components lie within a sector where the magnitude ratio and the phase difference show variability among different subjects, meaning that walking and running are closely related by their own unique mapping. That is, walking and running are not only distinguished by their kinematics (the range of movement made), but also significantly related by their kinetics (the forces that cause the movement). This also justifies the inclusion of the phase component in creating the gait signature and validates the assumption of phase modulation. (a) (b) Fig. 3: Phasor plot for the T1 of the (a) thigh and (b) lower leg; different symbols represent different subjects. Recall that the gait signatures are created from the PWM, and hence the PWM for running, PWM'R, can be deduced from the phase and magnitude of walking signal (φ W and MW), as () ( ) T W T W R M M φ + φ • • = ′ M PW   (10) Similarly, the PWM for walking, PWM'W, can be deduced from the phase and magnitude of running(φ R and MR), as () ( ) WR T R T MM φφ ′ =• − PWM   (11) In contrast to the PWM, which is irreversible, the mapping, T, contains information directly related to both gait and is reversible, as such the features of the motion within both gait are retained. The ability to describe the relationship in terms of phase and magnitude provides different information to enhance the signature and compensate for lost information. By means of the mapping, the gait signature can be made invariant and can be transformed from one form to another without losing the discriminatory ability.  5. Evaluation of the Mapping The database analysed here consists of the fronto-parallel views of 20 subjects walking and running on a motorised treadmill at their preferred speeds in their own choice of clothing. There are 6 samples of walking and running of each subject. The classification process is carried out via k-nearest neighbour and cross-validated by the leave-one-out rule. No doubt a more sophisticated classifier would be prudent, but the interest here is to evaluate the genuine discriminatory ability of the features. The signatures for walking are derived using each individual's unique mapping, based on the signature of running obtained from the feature extraction process. The derived signatures for walking are then tested on the database of walking subjects and similarly for running. The results shown in Table 1 are promising and the process involved implies the mapping offers invariance to signatures of different gait mode. The smooth variation with increase in k suggests that the data modelling approaches could be deployed to good effect here. Deduced Signature using T  k   Walking (%)  Running (%) 1 85.0  85.8 3 75.4  76.7 5 63.1  67.8 Table 1: Results of classification using T   Next, the mapping T, was used for classification alone with results shown in Table 2. The perfect classification rates (at least in this case) signify that the mapping is highly unique across the population and could be used as a compressed form of signature alone. The mapping is plotted in Fig. 4 for 13 subjects using spherical polar co-ordinates for 2 measures of phase and one of magnitude, superimposed on a unit sphere to aid visualisation, where different symbols represent different subjects. The dispersion of clusters confirms the high inter-class variability and low intra-class variability.    4The Mapping               Fig. 4: Part of the mapping in 3-D.   Walking (%)  Running (%)   k  T Original Enhanced Original  Enhanced1 100.0  84.2  95.8  91.7  98.3 3 100.0  67.5  89.2  84.2  92.9 5 100.0  55.0  74.4  68.9  82.5 Table 2: Results of classification using original and enhanced gait signatures. Finally the idea of deploying T in enhancing the original gait signature shows again how the invariant mapping contains more descriptive information. Recall that the original signal comprises the lower PWMs and T is added to the signature vector to yield the enhanced signature. As depicted in Table 2, the classification rate using the enhanced signature improves over the original signature. The clusters are more well-defined in the enhanced signature space, as the rates do not fall as rapidly as when using the original gait signature whilst k increases. These show that T can enhance the gait signatures. 6. Conclusions and Future Work A motion model invariant to walking and running has been deployed via computer vision techniques to capture the leg motion when walking and running. Fourier analysis of the motion provides useful features for recognition and serves as the basis for the analysis of the relationship between walking and running. The new findings and contributions are: •  A relationship between walking and running does exist and can be described by generic formulae. •  Walking and running can be assumed to be related by phase modulation.  •  Running is a more potent cue for recognition as this gait has more variability across the population. •  The mappings are highly unique: not only can they be used for recognition alone, but also to make signatures invariant and to provide means to transform from one gait mode to another. •  The mapping can capture the motion features for both gait and can then enhance the original gait signatures. Naturally, we shall in future aim to determine more precisely the nature of the mapping between running and walking, and whether it can be modelled. The understanding of the relationship between walking and running gait not only strengthens the automatic person identification system by gait (biometrics), but also may play an important role in other areas e.g. biomechanics, robotics and computer graphics animation. This novel idea particularly favours the computer animation of human locomotion because the analytical model, which needs relatively few parameters, is invariant to both walking and running gait, and the mapping offers a convenient morph from one gait mode to another. Equally, these advantages could benefit the field of bipedal robotics by assisting the development of an efficient gait algorithm. We intend to further investigate the mapping, the relationship of these two biomechanically distinct gait, the gait of twins, and its advantages in computer graphics and animation. Furthermore, similar idea could be deployed in motion analysis when walking on a treadmill and on the floor, or across different terrains.  References [1] J.K. Aggrawal and Q. Cai, Human Motion Analysis: A Review, CVIU, 73(3), pp 428-440, 1999. [2] G. Mather and L. Murdoch, Gender Discrimination in Biological Motion Displays based on Dynamic Cues, Proc. R. Soc. London, B: 258, pp 273-279, 1994. [3] J.W. Davis, Visual Categorization of Children and Adult Walking Styles, Proc. 3rd AVBPA 2001, pp 295-300, June 2001. [4] J.D. Shutler and M.S. Nixon, Zernike Velocity Moments for Description and Recognition of Moving Shapes, Proc. BMVC 2001 pp 705-714 , September 2001. [5] A.Y. Johnson and A.F. Bobick, A Multi-view Method for Gait Recognition Using Static Body Parameters, Proc.  3rd AVBPA 2001, pp 312-317, June 2001. [6] S.A. Niyogi and E.H. Adelson. Analyzing and Recognizing Walking Figures in XYT. CVPR, pp 469-474, 1994. [7] D. Cunado, M.S. Nixon, J.N. Carter, Automatic Gait Recognition via Model-based Evidence Gathering, Proc. AutoID'99, pp 27-30, 1999. [8] C.Y. Yam, M.S. Nixon and J.N. Carter, Extended Model-Based Automatic Gait Recognition of Walking and Running, Proc. 3rd AVBPA 2001, pp 278-283, June 2001. [9] C.Y. Yam, M.S. Nixon and J.N. Carter, Gait Recognition By Walking and Running: A Model-Based Approach, Accepted for ACCV 2002. [10] A.K. Jain, R. Bolle and S. Pankanti, Biometrics – Personal Identification in Networked Society. Kluwer, 1999. [11] V.T. Inman, H.J. Ralston and F. Todd, Human Walking. Williams & Wilkins, 1981. Acknowledgements We gratefully acknowledge partial support by the European Research Office of the US Army under Contract No. N68171-01-C-9002. 

A Multi-view Method for Gait Recognition Using Static Body Parameters,

Analyzing and Recognizing Walking Figures

Automatic Gait Recognition via Model-based Evidence Gathering,

Biometrics – Personal Identification in Networked Society.

Extended ModelBased Automatic Gait Recognition of Walking and Running,

Gait Recognition By Walking and Running: A Model-Based Approach, Accepted for ACCV

Gender Discrimination in Biological Motion Displays based on Dynamic Cues,

Human Motion Analysis:

Visual Categorization of Children and Adult Walking Styles,

Zernike Velocity Moments for Description and Recognition of Moving Shapes,

On the Relationship of Human Walking and Running: Automatic Person Identification by Gait

On the Relationship of Human Walking and Running: Automatic Person Identification by Gait

Abstract

Similar works

Full text

Available Versions

Southampton (e-Prints Soton)