Activity-conditioned continuous human pose estimation for performance analysis of athletes using the example of swimming

In this paper we consider the problem of human pose estimation in real-world videos of swimmers. Swimming channels allow filming swimmers simultaneously above and below the water surface with a single stationary camera. These recordings can be used to quantitatively assess the athletes' performance. The quantitative evaluation, so far, requires manual annotations of body parts in each video frame. We therefore apply the concept of CNNs in order to automatically infer the required pose information. Starting with an off-the-shelf architecture, we develop extensions to leverage activity information - in our case the swimming style of an athlete - and the continuous nature of the video recordings. Our main contributions are threefold: (a) We apply and evaluate a fine-tuned Convolutional Pose Machine architecture as a baseline in our very challenging aquatic environment and discuss its error modes, (b) we propose an extension to input swimming style information into the fully convolutional architecture and (c) modify the architecture for continuous pose estimation in videos. With these additions we achieve reliable pose estimates with up to +16% more correct body joint detections compared to the baseline architecture.Comment: 10 pages, 9 figures, accepted at WACV 201

SwimmerNET: Underwater 2D Swimmer Pose Estimation Exploiting Fully Convolutional Neural Networks

Professional swimming coaches make use of videos to evaluate their athletes' performances. Specifically, the videos are manually analyzed in order to observe the movements of all parts of the swimmer's body during the exercise and to give indications for improving swimming technique. This operation is time-consuming, laborious and error prone. In recent years, alternative technologies have been introduced in the literature, but they still have severe limitations that make their correct and effective use impossible. In fact, the currently available techniques based on image analysis only apply to certain swimming styles; moreover, they are strongly influenced by disturbing elements (i.e., the presence of bubbles, splashes and reflections), resulting in poor measurement accuracy. The use of wearable sensors (accelerometers or photoplethysmographic sensors) or optical markers, although they can guarantee high reliability and accuracy, disturb the performance of the athletes, who tend to dislike these solutions. In this work we introduce swimmerNET, a new marker-less 2D swimmer pose estimation approach based on the combined use of computer vision algorithms and fully convolutional neural networks. By using a single 8 Mpixel wide-angle camera, the proposed system is able to estimate the pose of a swimmer during exercise while guaranteeing adequate measurement accuracy. The method has been successfully tested on several athletes (i.e., different physical characteristics and different swimming technique), obtaining an average error and a standard deviation (worst case scenario for the dataset analyzed) of approximately 1 mm and 10 mm, respectively

Hubungan Berat Badan dan Tinggi Badan Dengan Kecepatan Renang Gaya Bebas 50 Meter Pada Atlet Renang Noren Tirta Buana (NTB)

  This study aims to determine the relationship between body weight and height with the speed of 50-meter freestyle swimming in Noren Tirta Buana (NTB) swimming athletes. The research method used in this study is the correlation method with data collection techniques using tests and measurements. The population and sample in this study were athletes Noren Tirta Buana (NTB) with a total sampling technique so that the sample was 15 people. The results of this study show that there is a significant relationship between body weight and swimming speed in the 50-meter crawl style for athletes Noren Tirta Buana”; there is an important relationship between body height and swimming speed in the 50-meter crawl style for athletes Noren Tirta Buana”; there is a significant relationship between body weight and size on the swimming speed of the 50-meter crawl style in female athletes 50 meters in athlete Noren Tirta Buana. In conclusion, there is a significant relationship between body weight and height in freestyle swimming speed in Noren Tirta Buana (NTB) swimming athletes. Keywords: Height, Swimming, Weight

Recognition of freely selected keypoints on human limbs

Nearly all Human Pose Estimation (HPE) datasets consist of a fixed set of keypoints. Standard HPE models trained on such datasets can only detect these keypoints. If more points are desired, they have to be manually annotated and the model needs to be retrained. Our approach leverages the Vision Transformer architecture to extend the capability of the model to detect arbitrary keypoints on the limbs of persons. We propose two different approaches to encode the desired keypoints. (1) Each keypoint is defined by its position along the line between the two enclosing keypoints from the fixed set and its relative distance between this line and the edge of the limb. (2) Keypoints are defined as coordinates on a norm pose. Both approaches are based on the TokenPose architecture, while the keypoint tokens that correspond to the fixed keypoints are replaced with our novel module. Experiments show that our approaches achieve similar results to TokenPose on the fixed keypoints and are capable of detecting arbitrary keypoints on the limbs.Comment: accepted at CVSports (CVPR 2022 Workshops

Detecting arbitrary keypoints on limbs and skis with sparse partly correct segmentation masks

Analyses based on the body posture are crucial for top- class athletes in many sports disciplines. If at all, coaches label only the most important keypoints, since manual anno- tations are very costly. This paper proposes a method to de- tect arbitrary keypoints on the limbs and skis of professional ski jumpers that requires a few, only partly correct segmen- tation masks during training. Our model is based on the Vision Transformer architecture with a special design for the input tokens to query for the desired keypoints. Since we use segmentation masks only to generate ground truth labels for the freely selectable keypoints, partly correct seg- mentation masks are sufficient for our training procedure. Hence, there is no need for costly hand-annotated segmen- tation masks. We analyze different training techniques for freely selected and standard keypoints, including pseudo la- bels, and show in our experiments that only a few partly cor- rect segmentation masks are sufficient for learning to detect arbitrary keypoints on limbs and skis

Motion-DVAE: Unsupervised learning for fast human motion denoising

Pose and motion priors are crucial for recovering realistic and accurate human motion from noisy observations. Substantial progress has been made on pose and shape estimation from images, and recent works showed impressive results using priors to refine frame-wise predictions. However, a lot of motion priors only model transitions between consecutive poses and are used in time-consuming optimization procedures, which is problematic for many applications requiring real-time motion capture. We introduce Motion-DVAE, a motion prior to capture the short-term dependencies of human motion. As part of the dynamical variational autoencoder (DVAE) models family, Motion-DVAE combines the generative capability of VAE models and the temporal modeling of recurrent architectures. Together with Motion-DVAE, we introduce an unsupervised learned denoising method unifying regression- and optimization-based approaches in a single framework for real-time 3D human pose estimation. Experiments show that the proposed approach reaches competitive performance with state-of-the-art methods while being much faster

Towards using artificial intelligence as tool in artistic gymnastics coaching : case backward giant circle

The objective of this thesis was to study whether it is possible to create a system that estimates artistic gymnast’s body joint angles based on a low-budget 2-dimensional single RGB video recording. To meet the objective, 54 video files were collected on gymnasts performing backward giant circle skill, together with assessments of the performances by two professional coaches. The video files contained total of 233 repetitions of the skill. A pilot system of computer vision algorithms was developed, using an open source human body pose recognition algorithm. An algorithm based on pixel grayscale value was developed and used to recognize starting and ending moment of a repetition and to sample each repetition at 7 key phases. Body joint angle estimates were calculated based on the body part location estimates of the 1631 samples. The work proved that it is possible to develop a system that estimates body joint angles of an artistic gymnast. It was found that rotation and cropping of the frames improved probability of yielding correct estimates. The angle estimate for knees had highest, up to 66%, correlation with coach evaluations. Hips and shoulders had weak but significant correlation with coach evaluations. The results indicate that it is possible to develop a low-budget system that could work as augmented tool in artistic gymnastics coaching. In addition, human body pose recognition provides a new method to biomechanical research of artistic gymnastics