Classification of skateboarding tricks by synthesizing transfer learning models and machine learning classifiers using different input signal transformations

Skateboarding has made its Olympic debut at the delayed Tokyo 2020 Olympic Games. Conventionally, in the competition scene, the scoring of the game is done manually and subjectively by the judges through the observation of the trick executions. Nevertheless, the complexity of the manoeuvres executed has caused difficulties in its scoring that is obviously prone to human error and bias. Therefore, the aim of this study is to classify five skateboarding flat ground tricks which are Ollie, Kickflip, Shove-it, Nollie and Frontside 180. This is achieved by using three optimized machine learning models of k-Nearest Neighbor (kNN), Random Forest (RF), and Support Vector Machine (SVM) from features extracted via eighteen transfer learning models. Six amateur skaters performed five tricks on a customized ORY skateboard. The raw data from the inertial measurement unit (IMU) embedded on the developed device attached to the skateboarding were extracted. It is worth noting that four types of input images were transformed via Fast Fourier Transform (FFT), Continuous Wavelet Transform (CWT), Discrete Wavelet Transform (DWT) and synthesized raw image (RAW) from the IMU-based signals obtained. The optimized form of the classifiers was obtained by performing GridSearch optimization technique on the training dataset with 3-folds cross-validation on a data split of 4:1:1 ratio for training, validation and testing, respectively from 150 transformed images. It was shown that the CWT and RAW images used in the MobileNet transfer learning model coupled with the optimized SVM and RF classifiers exhibited a test accuracy of 100%. In order to identify the best possible method for the pipelines, computational time was used to evaluate the various models. It was concluded that the RAW-MobileNet-optimized-RF approach was the most effective one, with a computational time of 24.796875 seconds. The results of the study revealed that the proposed approach could improve the classification of skateboarding tricks

The classification of skateboarding trick images by means of transfer learning and machine learning models

The evaluation of tricks executions in skateboarding is commonly executed manually and subjectively. The panels of judges often rely on their prior experience in identifying the effectiveness of tricks performance during skateboarding competitions. This technique of classifying tricks is deemed as not a practical solution for the evaluation of skateboarding tricks mainly for big competitions. Therefore, an objective and unbiased means of evaluating skateboarding tricks for analyzing skateboarder’s trick is nontrivial. This study aims at classifying flat ground tricks namely Ollie, Kickflip, Pop Shove-it, Nollie Frontside Shove-it, and Frontside 180 through the camera vision and the combination of Transfer Learning (TL) and Machine Learning (ML). An amateur skateboarder (23 years of age with ± 5.0 years’ experience) executed five tricks for each type of trick repeatedly on an HZ skateboard from a YI action camera placed at a distance of 1.26 m on a cemented ground. The features from the image obtained are extracted automatically via 18 TL models. The features extracted from the models are then fed into different tuned ML classifiers models, for instance, Support Vector Machine (SVM), k-Nearest Neighbors (k-NN), and Random Forest (RF). The grid search optimization technique through five-fold cross-validation was used to tune the hyperparameters of the classifiers evaluated. The data (722 images) was split into training, validation, and testing with a stratified ratio of 60:20:20, respectively. The study demonstrated that VGG16 + SVM and VGG19 + RF attained classification accuracy (CA) of 100% and 98%, respectively on the test dataset, followed by VGG19 + k-NN and also DenseNet201 + k-NN that achieved a CA of 97%. In order to evaluate the developed pipelines, robustness evaluation was carried out via the form of independent testing that employed the augmented images (2250 images). It was found that VGG16 + SVM, VGG19 + k-NN, and DenseNet201 + RF (by average) are able to yield reasonable CA with 99%, 98%, and 97%, respectively. Conclusively, based on the robustness evaluation, it can be ascertained that the VGG16 + SVM pipeline able to classify the tricks exceptionally well. Therefore, from the present study, it has been demonstrated that the proposed pipelines may facilitate judges in providing a more accurate evaluation of the tricks performed as opposed to the traditional method that is currently applied in competitions

The classification of skateboarding tricks by means of support vector machine: An evaluation of significant time-domain features

This study aims to improve classification accuracy of different Support Vector Machine (SVM) models in classifying flat ground tricks namely Ollie, Kick-flip, Shove-it, Nollie and Frontside 180 through the identification of significant time-domain features. An amateur skateboarder (23 years of age ±5.0 years’ experience) executed five tricks for each type of trick repeatedly on a customized ORY skateboard (IMU sensor fused) on a cemented ground. From the IMU data a total of 36 features were extracted through statistical measures. The significant features were identified through two feature selection methods, namely Pearson and Chi-Squared. The variation of the SVM models (kernel-based) was evaluated both on all features and selected features in classifying the skateboarding tricks. It was shown from the study that all classifiers improved significantly in terms of training accuracy, prediction speed, training time and test accuracy. The Cubic-based SVM and Quadratic-based SVM demonstrated a 100% accuracy on both the test and train dataset, however, the Cubic-based SVM model provided the fastest training time and prediction speed between the two models. It could be concluded that the proposed method is able to improve the classification of the skateboarding tricks well

The classification of skateboarding trick manoeuvres: A K-nearest neighbour approach

The evaluation of skateboarding tricks is commonly carried out subjectively through the prior experience of the panel of judges during skateboarding competitions. Hence, this technique evaluation is often impartial to a certain degree. This study aims at classifying flat ground tricks namely Ollie, Kickflip, Shove-it, Nollie and Frontside 180 through the use of Inertial Measurement Unit (IMU) and a class of machine learning model namely k-Nearest Neighbour (k-NN). An amateur skateboarder (23 years of age ± 5.0 years’ experience) executed five tricks for each type of trick repeatedly on a customized ORY skateboard (IMU sensor fused) on a cemented ground. A number of features were extracted and engineered from the IMU data, i.e., mean, skewness, kurtosis, peak to peak, root mean square as well as standard deviation of the acceleration and angular velocities along the primary axes. A variation of k-NN algorithms were tested based on the number of neighbours, as well as the weight and the type of distance metric used. It was shown from the present preliminary investigation, that the k-NN model which employs k = 1 with an equal weight applied to the Euclidean distance metric yielded a classification accuracy of 85%. Therefore, it could be concluded that the proposed method is able to classify the skateboard tricks reasonably well and will in turn, assist the judges in providing more accurate evaluation of the tricks as opposed to the conventional-subjective based assessment that is applied at presen

The classification of skateboarding tricks : A transfer learning and machine learning approach

The skateboarding scene has arrived at new statures, particularly with its first appearance at the now delayed Tokyo Summer Olympic Games. Hence, attributable to the size of the game in such competitive games, progressed creative appraisal approaches have progressively increased due consideration by pertinent partners, particularly with the enthusiasm of a more goal-based assessment. This study purposes for classifying skateboarding tricks, specifically Frontside 180, Kickflip, Ollie, Nollie Front Shove-it, and Pop Shove-it over the integration of image processing, Trasnfer Learning (TL) to feature extraction enhanced with tradisional Machine Learning (ML) classifier. A male skateboarder performed five tricks every sort of trick consistently and the YI Action camera captured the movement by a range of 1.26 m. Then, the image dataset were features built and extricated by means of three TL models, and afterward in this manner arranged to utilize by k-Nearest Neighbor (k-NN) classifier. The perception via the initial experiments showed, the MobileNet, NASNetMobile, and NASNetLarge coupled with optimized k-NN classifiers attain a classification accuracy (CA) of 95%, 92% and 90%, respectively on the test dataset. Besides, the result evident from the robustness evaluation showed the MobileNet+k-NN pipeline is more robust as it could provide a decent average CA than other pipelines. It would be demonstrated that the suggested study could characterize the skateboard tricks sufficiently and could, over the long haul, uphold judges decided for giving progressively objective-based decision

The classification of skateboarding tricks via transfer learning pipelines

This study aims at classifying flat ground tricks, namely Ollie, Kickflip, Shove-it, Nollie and Frontside 180, through the identification of significant input image transformation on different transfer learning models with optimized Support Vector Machine (SVM) classifier. A total of six amateur skateboarders (20 ± 7 years of age with at least 5.0 years of experience) executed five tricks for each type of trick repeatedly on a customized ORY skateboard (IMU sensor fused) on a cemented ground. From the IMU data, a total of six raw signals extracted. A total of two input image type, namely raw data (RAW) and Continous Wavelet Transform (CWT), as well as six transfer learning models from three different families along with gridsearched optimized SVM, were investigated towards its efficacy in classifying the skateboarding tricks. It was shown from the study that RAW and CWT input images on MobileNet, MobileNetV2 and ResNet101 transfer learning models demonstrated the best test accuracy at 100% on the test dataset. Nonetheless, by evaluating the computational time amongst the best models, it was established that the CWTMobileNet-Optimized SVM pipeline was found to be the best. It could be concluded that the proposed method is able to facilitate the judges as well as coaches in identifying skateboarding tricks executio

Generalized and efficient skill assessment from IMU data with applications in gymnastics and medical training

Human activity recognition is progressing from automatically determining what a person is doing and when, to additionally analyzing the quality of these activities—typically referred to as skill assessment. In this chapter, we propose a new framework for skill assessment that generalizes across application domains and can be deployed for near-real-time applications. It is based on the notion of repeatability of activities defining skill. The analysis is based on two subsequent classification steps that analyze (1) movements or activities and (2) their qualities, that is, the actual skills of a human performing them. The first classifier is trained in either a supervised or unsupervised manner and provides confidence scores, which are then used for assessing skills. We evaluate the proposed method in two scenarios: gymnastics and surgical skill training of medical students. We demonstrate both the overall effectiveness and efficiency of the generalized assessment method, especially compared to previous work

IMU Data-based Recognition for Sports Exercises: An Enhanced Distance Optimization Approach for Repetition Counting ​Across Activities

In the field of human activity recognition (HAR), inertial measurement units (IMUs) are a commonly used method to record movement patterns. In the study presented in this paper, IMU captured seven different sports exercises performed by 21 participants. In the preliminary data analysis phase, an exercise classification was conducted using the Long Short Term Memory (LSTM) Network and Temporal Convolutional Network (TCN). The LSTM achieved an accuracy rate of 94.2 % for training and 90.8 % for testing. Similarly, the TCN demonstrated rates of 95.5 % for training and 91.6 % for testing. The subsequent stage was centered on quantifying the number of completed repetitions. A distance value was derived which showed promising results for exercise-independent counting without the need for manual feature selection. For further improvement, a range-to-mean ratio of the standard deviation was calculated and used for feature selection. Combined with a local extrema analysis of the modified distance values the accuracy of counting repetitions was significantly improved, especially for exercises that show irregularities in the signal course