SPACE-TIME GRAPH-BASED CONVOLUTIONAL NEURAL NETWORKS OF STUDY ON MOVEMENT RECOGNITION OF FOOTBALL PLAYERS

Behaviour recognition technology is an interdisciplinary technology, integrating many research achievements in computer vision, deep learning, pattern recognition and other fields. The key information of bone data on human behavior can not only accurately describe the motion posture of the human body in three-dimensional space, but also its rigid connection structure is robust to various external interference factors. However, the behavioral recognition algorithm is influenced by different factors such as background, light and environment, which is easy to lead to unstable recognition accuracy and limited application scenarios. To address this problem, in this paper, we propose a noise filtering algorithm based on data correlation and skeleton energy model filtering, construct a set of football player data sets, using the ST-GCN algorithm to train the skeleton characteristics of football players, and construct a behavior recognition system applied to football players. Finally, by comparing the accuracy of Deep LSTM, 2s-AGCN and the algorithm in this paper, the accuracy of TOP1 and TOP5 is 39.97% and 66.34%, respectively, which are significantly higher than the other two algorithms. It can realize the statistics of athletes and analyze the technical and tactical movements of players on the football field

Estimating Player Positions from Padel High-Angle Videos: Accuracy Comparison of Recent Computer Vision Methods

The estimation of player positions is key for performance analysis in sport. In this paper, we focus on image-based, single-angle, player position estimation in padel. Unlike tennis, the primary camera view in professional padel videos follows a de facto standard, consisting of a high-angle shot at about 7.6 m above the court floor. This camera angle reduces the occlusion impact of the mesh that stands over the glass walls, and offers a convenient view for judging the depth of the ball and the player positions and poses. We evaluate and compare the accuracy of state-of-the-art computer vision methods on a large set of images from both amateur videos and publicly available videos from the major international padel circuit. The methods we analyze include object detection, image segmentation and pose estimation techniques, all of them based on deep convolutional neural networks. We report accuracy and average precision with respect to manually-annotated video frames. The best results are obtained by top-down pose estimation methods, which offer a detection rate of 99.8% and a RMSE below 5 and 12 cm for horizontal/vertical court-space coordinates (deviations from predicted and ground-truth player positions). These results demonstrate the suitability of pose estimation methods based on deep convolutional neural networks for estimating player positions from single-angle padel videos. Immediate applications of this work include the player and team analysis of the large collection of publicly available videos from international circuits, as well as an inexpensive method to get player positional data in amateur padel clubs.This work has been partially funded by the Spanish Ministry of Economy and Competitiveness and FEDER under grant TIN2017-88515-C2-1-R

Perception tools and their usage in sports

Computer vision and perception tools are an important step on the developing world of robotics. The goal of computer vision development is to enable machines to understand visual information. When this goal is acquired, there are countless usages in fields such as production, healthcare, and entertainment like sports. This thesis addresses perception tools, computer vision and especially their usage amongst sports. The thesis can be roughly divided into two parts, the first part is literature research about computer vision and perception tools. The second part of the thesis is a practical example with OpenDR’s pose estimator tool, which is used to do a short analysis on two separate golf swings. These swings are then compared to each other and by this one way of using perception tools in sports is showcased. First part of the literature research addresses the basics of computer vision. The beginning of this part introduces what computer vision and perception tools are. In addition, a rough version of a vision system is presented. After that we move on to how computers process images. Briefly computers segment the picture and then give a value to each part that corresponds into a color. After image processing we switch over to 3D-vision and present three different ways on how it is executed. The methods that were presented are, triangulation, time-of-flight, and structured light. Lastly, we researched on how object tracking is implemented on vision systems. We found out that no one way of executing object tracking is set in stone, and there are available several different algorithms. One common factor in these algorithms was the need of background filtering. Usually this is done by comparing subsequent images and separating objects that are in motion from those that are motionless. At the end of this part project OpenDR is also showcased, which is used in the practical part of the thesis. The second part of the literature research contains presentations of different systems that are already used in sports. In total 4 systems were presented, each of them focusing on a different part of computer vision. The presented systems were used in hockey, tennis, cricket, and badminton. The purpose of these systems include improvement in training efficiency, added entertainment value, and increasing the accuracy of officiating. Some of the systems are already in commercial use and some were still in development. Although the purpose of these systems is to develop sports in general, there are some difficulties. These difficulties include technical ones like accuracy of the system or price, and impact on the entertainment value by disrupting the flow of the game. On the last part of the thesis a pose estimator algorithm was used to detect joints and other keypoints from images. These keypoints were used to compare two different golf swings and to detect differences in these swings. These differences were then used to consider on what should be changed on the swing to acquire better results. The algorithm made it easier to detect the differences in the swings and we concluded that with more data such as different angles, in addition to better sport knowledge, more accurate advice could be provided

Fast human behavior analysis for scene understanding

Human behavior analysis has become an active topic of great interest and relevance for a number of applications and areas of research. The research in recent years has been considerably driven by the growing level of criminal behavior in large urban areas and increase of terroristic actions. Also, accurate behavior studies have been applied to sports analysis systems and are emerging in healthcare. When compared to conventional action recognition used in security applications, human behavior analysis techniques designed for embedded applications should satisfy the following technical requirements: (1) Behavior analysis should provide scalable and robust results; (2) High-processing efficiency to achieve (near) real-time operation with low-cost hardware; (3) Extensibility for multiple-camera setup including 3-D modeling to facilitate human behavior understanding and description in various events. The key to our problem statement is that we intend to improve behavior analysis performance while preserving the efficiency of the designed techniques, to allow implementation in embedded environments. More specifically, we look into (1) fast multi-level algorithms incorporating specific domain knowledge, and (2) 3-D configuration techniques for overall enhanced performance. If possible, we explore the performance of the current behavior-analysis techniques for improving accuracy and scalability. To fulfill the above technical requirements and tackle the research problems, we propose a flexible behavior-analysis framework consisting of three processing-layers: (1) pixel-based processing (background modeling with pixel labeling), (2) object-based modeling (human detection, tracking and posture analysis), and (3) event-based analysis (semantic event understanding). In Chapter 3, we specifically contribute to the analysis of individual human behavior. A novel body representation is proposed for posture classification based on a silhouette feature. Only pure binary-shape information is used for posture classification without texture/color or any explicit body models. To this end, we have studied an efficient HV-PCA shape-based descriptor with temporal modeling, which achieves a posture-recognition accuracy rate of about 86% and outperforms other existing proposals. As our human motion scheme is efficient and achieves a fast performance (6-8 frames/second), it enables a fast surveillance system or further analysis of human behavior. In addition, a body-part detection approach is presented. The color and body ratio are combined to provide clues for human body detection and classification. The conventional assumption of up-right body posture is not required. Afterwards, we design and construct a specific framework for fast algorithms and apply them in two applications: tennis sports analysis and surveillance. Chapter 4 deals with tennis sports analysis and presents an automatic real-time system for multi-level analysis of tennis video sequences. First, we employ a 3-D camera model to bridge the pixel-level, object-level and scene-level of tennis sports analysis. Second, a weighted linear model combining the visual cues in the real-world domain is proposed to identify various events. The experimentally found event extraction rate of the system is about 90%. Also, audio signals are combined to enhance the scene analysis performance. The complete proposed application is efficient enough to obtain a real-time or near real-time performance (2-3 frames/second for 720×576 resolution, and 5-7 frames/second for 320×240 resolution, with a P-IV PC running at 3GHz). Chapter 5 addresses surveillance and presents a full real-time behavior-analysis framework, featuring layers at pixel, object, event and visualization level. More specifically, this framework captures the human motion, classifies its posture, infers the semantic event exploiting interaction modeling, and performs the 3-D scene reconstruction. We have introduced our system design based on a specific software architecture, by employing the well-known "4+1" view model. In addition, human behavior analysis algorithms are directly designed for real-time operation and embedded in an experimental runtime AV content-analysis architecture. This executable system is designed to be generic for multiple streaming applications with component-based architectures. To evaluate the performance, we have applied this networked system in a single-camera setup. The experimental platform operates with two Pentium Quadcore engines (2.33 GHz) and 4-GB memory. Performance evaluations have shown that this networked framework is efficient and achieves a fast performance (13-15 frames/second) for monocular video sequences. Moreover, a dual-camera setup is tested within the behavior-analysis framework. After automatic camera calibration is conducted, the 3-D reconstruction and communication among different cameras are achieved. The extra view in the multi-camera setup improves the human tracking and event detection in case of occlusion. This extension of multiple-view fusion improves the event-based semantic analysis by 8.3-16.7% in accuracy rate. The detailed studies of two experimental intelligent applications, i.e., tennis sports analysis and surveillance, have proven their value in several extensive tests in the framework of the European Candela and Cantata ITEA research programs, where our proposed system has demonstrated competitive performance with respect to accuracy and efficiency

Assessment of Physical Fitness and Training Effect in Individual Sports

Physical fitness is the basis for the success of players in sports, and its monitoring makes it possible to assess the effectiveness of training and identify possible errors. During training, thanks to the use of control results, these activities are modified, which better prepares players for competition. This Special Issue, entitled "Assessment of Physical Fitness and the Effect of Training in Individual Sports" presents the results of coaching control and the results of monitoring progression in training, as well as an assessment of the physical fitness of athletes practicing individual sports

Multi-sensor human action recognition with particular application to tennis event-based indexing

The ability to automatically classify human actions and activities using vi- sual sensors or by analysing body worn sensor data has been an active re- search area for many years. Only recently with advancements in both fields and the ubiquitous nature of low cost sensors in our everyday lives has auto- matic human action recognition become a reality. While traditional sports coaching systems rely on manual indexing of events from a single modality, such as visual or inertial sensors, this thesis investigates the possibility of cap- turing and automatically indexing events from multimodal sensor streams. In this work, we detail a novel approach to infer human actions by fusing multimodal sensors to improve recognition accuracy. State of the art visual action recognition approaches are also investigated. Firstly we apply these action recognition detectors to basic human actions in a non-sporting con- text. We then perform action recognition to infer tennis events in a tennis court instrumented with cameras and inertial sensing infrastructure. The system proposed in this thesis can use either visual or inertial sensors to au- tomatically recognise the main tennis events during play. A complete event retrieval system is also presented to allow coaches to build advanced queries, which existing sports coaching solutions cannot facilitate, without an inordi- nate amount of manual indexing. The event retrieval interface is evaluated against a leading commercial sports coaching tool in terms of both usability and efficiency