Our work using generative deep learning models to generate synthetic human movement data to augment existing datasets was presented at the 9th World Congress of Biomechanics

Bicer, M

Modenese, L

Phillips, A

English

Spiral - Imperial College Digital Repository

Generative deep learning applied to biomechanics: creating an infinite number of realistic walking data for modelling and data augmentation purposes  Metin Bicer1, Andrew TM Phillips1, Luca Modenese1  1Department of Civil and Environmental Engineering, Imperial College London, UK  Introduction The performance of deep learning models can be improved by augmenting the available data, e.g., cropping and rotating images, or generating synthetic data using Generative Adversarial Networks (GANs) [1]. Only the second technique is applicable to biomechanical data such as marker trajectories and ground reaction force (GRF). Thus, we aimed to show that GANs can enhance dataset sizes by generating synthetic but realistic marker and GRF data characterizing healthy gait trials. To the best of the authors’ knowledge, this is the first generative deep learning approach in biomechanics.  Methods We used an adversarial autoencoder [2] composed of 1) an encoder used to map real samples to latent vectors, 2) a decoder to reproduce the given inputs from encoded samples, and 3) a discriminator to predict whether a vector belonged to the encoded samples set or was selected randomly from a normal distribution. Our dataset included eight subjects performing 55 walking trials cropped to a single gait cycle. Marker trajectories and GRFs were transformed to 3-channel tensors. The network trained with this dataset is called the core model. The same network was also trained with an extended dataset (extended model), including joint angles calculated by inverse kinematics (IK) using a kinematic model [3]. After proper training, each model’s generator was used to generate 100 synthetic trials. We assessed the similarity of the generated data to the real data using joint angles calculated from the marker trajectories by direct kinematics (DK), GRFs, and joint angles and moments calculated by IK and inverse dynamics (ID) in OpenSim [4]. Significant differences from a 1D two-tailed paired t-test (α=0.05) [4] between the kinematics and kinetics waveforms of synthetic and real trials were reported as percentages of the gait cycle for the kinematics and stance phase for the kinetics.  Results The models successfully generated 100 distinct synthetic trials. The core model yielded negligible differences for DK-computed joint angles (0.36%) and GRFs (0.00%), but not for inverse methods (IK: 29.21%, ID:13.91%). The extended model’s synthetic trials showed no significant differences in the DK-computed joint angles and GRFs. The joint angles and moments calculated by IK (0.00%) and ID (7.09%) were remarkably improved. Representative synthetic and real trials, throughout a gait cycle, are presented in Figure 1.  Discussion The potential of GANs in biomechanics was demonstrated by generating a large number of synthetic healthy gait trials that were similar, but not identical, to the real trials. The extended model, which was implicitly aware of an underlying kinematic model, generated gait trials more similar to those of the original dataset than the core model. This approach will be used to augment training datasets for algorithms computing joint kinematics and kinetics from wearable sensors.   Acknowledgements Education Ministry of Turkey Scholarship to MB, Imperial College Research Fellowship to LM.  References 1. Goodfellow et al. Adv. Neural Inf. Process. Syst., 27, 2014. 2. Makhzani et al. arXiv:1511.05644, 2015. 3. Modenese et al. J. Biomech. 44:2185-2193, 2011. 4. Delp et al. IEEE Trans. Biomed. Eng. 54(11):1940-1950, 2007. 5. Pataky et al. J. Biomech. 46:2394-2401, 2013.    

Generative deep learning applied to biomechanics: creating an infinite number of realistic walking data for modelling and data augmentation purposes

http://spiral.imperial.ac.uk/bitstream/10044/1/101607/6/Bicer_et_at_WCB22_Generative.pdf

Generative deep learning applied to biomechanics: creating an infinite number of realistic walking data for modelling and data augmentation purposes

Abstract

Similar works

Full text

Available Versions

Spiral - Imperial College Digital Repository