Validity and reliability of NOTCH® inertial sensors for measuring elbow joint angle during tennis forehand at different sampling frequencies

Portable and low-cost motion capture systems are gaining importance for biomechanical analysis. The aim was to determine the concurrent validity and reliability of the NOTCH® inertial sensors to measure the elbow angle during tennis forehand at different sampling frequencies (100, 250 and 500 Hz), using an optical capture system with sub-millimetre accuracy as a reference. 15 competitive players performed forehands wearing NOTCH and an upper body marker-set and the signals from both systems were adjusted and synchronized. The error magnitude was tolerable (5-10◦) for all joint-axis and sampling frequencies, increasing significantly at 100 Hz for the flexion–extension and pronation-supination angles (p = 0.002 and 0.023; Cohen d > 0.8). Concordance correlation coefficient was very large (0.7–0.9) in all cases. The within-subject error variation between the test–retest did not show significant differences (p > 0.05). NOTCH® is a valid, reliable and portable alternative to measure elbow angles during tennis forehand

Optimal Stroke Learning with Policy Gradient Approach for Robotic Table Tennis

Learning to play table tennis is a challenging task for robots, as a wide variety of strokes required. Recent advances have shown that deep Reinforcement Learning (RL) is able to successfully learn the optimal actions in a simulated environment. However, the applicability of RL in real scenarios remains limited due to the high exploration effort. In this work, we propose a realistic simulation environment in which multiple models are built for the dynamics of the ball and the kinematics of the robot. Instead of training an end-to-end RL model, a novel policy gradient approach with TD3 backbone is proposed to learn the racket strokes based on the predicted state of the ball at the hitting time. In the experiments, we show that the proposed approach significantly outperforms the existing RL methods in simulation. Furthermore, to cross the domain from simulation to reality, we adopt an efficient retraining method and test it in three real scenarios. The resulting success rate is 98% and the distance error is around 24.9 cm. The total training time is about 1.5 hours

Motion Sensors-Based Human Behavior Recognition And Analysis

Human behavior recognition and analysis have been considered as a core technology that can facilitate a variety of applications. However, accurate detection and recognition of human behavior is still a big challenge that attracts a lot of research efforts. Among all the research works, motion sensors-based human behavior recognition is promising as it is low cost, low power, and easy to carry. In this dissertation, we use motion sensors to study human behaviors. First, we present Ultigesture (UG) wristband, a hardware platform for detecting and analyzing human behavior. The hardware platform integrates an accelerometer, gyroscope, and compass sensor, providing a combination of (1) fully open Application Programming Interface (API) for various application development, (2) appropriate form factor for comfortable daily wear, and (3) affordable cost for large scale adoption. Second, we study the hand gesture recognition problem when a user performs gestures continuously. we propose a novel continuous gesture recognition algorithm. It accurately and automatically separates hand movements into segments, and merges adjacent segments if needed, so that each gesture only exists in one segment. Then, we apply the Hidden Markov Model to classify each segment into one of predefined hand gestures. Experiments with human subjects show that the recognition accuracy is 99.4% when users perform gestures discretely, and 94.6% when users perform gestures continuously. Third, we study the hand gesture recognition problem when a user is moving. We propose a novel mobility-aware hand gesture segmentation algorithm to detect and segment hand gestures. We also propose a Convolutional Neural Network to classify hand gestures with mobility noises. For the leave-one-subject-out cross-validation test, experiments with human subjects show that the proposed segmentation algorithm achieves 94.0% precision, and 91.2% recall when the user is moving. The proposed hand gesture classification algorithm is 16.1%, 15.3%, and 14.4% more accurate than state-of-the-art work when the user is standing, walking, and jogging, respectively. Finally, we present a tennis ball speed estimation system, TennisEye, which uses a racket-mounted motion sensor to estimate ball speed. We divide the tennis shots into three categories: serve, groundstroke, and volley. For a serve, we propose a regression model to estimate the ball speed. In addition, we propose a physical model and a regression model for both groundstroke and volley shots. Under the leave-one-subject-out cross-validation test, evaluation results show that TennisEye is 10.8% more accurate than the state-of-the-art work

Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts

In this paper, we present a method for table tennis ball trajectory filtering and prediction. Our gray-box approach builds on a physical model. At the same time, we use data to learn parameters of the dynamics model, of an extended Kalman filter, and of a neural model that infers the ball's initial condition. We demonstrate superior prediction performance of our approach over two black-box approaches, which are not supplied with physical prior knowledge. We demonstrate that initializing the spin from parameters of the ball launcher using a neural network drastically improves long-time prediction performance over estimating the spin purely from measured ball positions. An accurate prediction of the ball trajectory is crucial for successful returns. We therefore evaluate the return performance with a pneumatic artificial muscular robot and achieve a return rate of 29/30 (97.7%).Comment: Accepted for publication at the 5th Annual Conference on Learning for Dynamics and Control (L4DC) 2023. With supplementary materia

Adaptive Robot Systems in Highly Dynamic Environments: A Table Tennis Robot

Hintergrund: Tischtennis bietet ideale Bedingungen, um Kamera-basierte Roboterarme am Limit zu testen. Die besondere Herausforderung liegt in der hohen Geschwindigkeit des Spiels und in der großen Varianz von Spin und Tempo jedes einzelnen Schlages. Die bisherige Forschung mit Tischtennisrobotern beschränkt sich jedoch auf einfache Szenarien, d.h. auf langsame Bälle mit einer geringen Rotation. Forschungsziel: Es soll ein lernfähiger Tischtennisroboter entwickelt werden, der mit dem Spin menschlicher Gegner umgehen kann. Methoden: Das vorgestellte Robotersystem besteht aus sechs Komponenten: Ballpositionserkennung, Ballspinerkennung, Balltrajektorienvorhersage, Schlagparameterbestimmung, Robotertrajektorienplanung und Robotersteuerung. Zuerst wird der Ball mit traditioneller Bildverarbeitung in den Kamerabildern lokalisiert. Mit iterativer Triangulation wird dann seine 3D-Position berechnet. Aus der Kurve der Ballpositionen wird die aktuelle Position und Geschwindigkeit des Balles ermittelt. Für die Spinerkennung werden drei Methoden präsentiert: Die ersten beiden verfolgen die Bewegung des aufgedruckten Ball-Logos auf hochauflösenden Bildern durch Computer Vision bzw. Convolutional Neural Networks. Im dritten Ansatz wird die Flugbahn des Balls unter Berücksichtigung der Magnus-Kraft analysiert. Anhand der Position, der Geschwindigkeit und des Spins des Balls wird die zukünftige Flugbahn berechnet. Dafür wird die physikalische Diffenzialgleichung mit Gravitationskraft, Luftwiderstandskraft und Magnus-Kraft schrittweise gelöst. Mit dem berechneten Zustand des Balls am Schlagpunkt haben wir einen Reinforcement-Learning-Algorithmus trainiert, der bestimmt, mit welchen Schlagparametern der Ball zu treffen ist. Eine passende Robotertrajektorie wird von der Reflexxes-Bibliothek generiert. %Der Roboter wird dann mit einer Frequenz von 250 Hz angesteuert. Ergebnisse: In der quantitativen Auswertung erzielen die einzelnen Komponenten mindestens so gute Ergebnisse wie vergleichbare Tischtennisroboter. Im Hinblick auf das Forschungsziel konnte der Roboter - ein Konterspiel mit einem Menschen führen, mit bis zu 60 Rückschlägen, - unterschiedlichen Spin (Über- und Unterschnitt) retournieren - und mehrere Tischtennisübungen innerhalb von 200 Schlägen erlernen. Schlußfolgerung: Bedeutende algorithmische Neuerungen führen wir in der Spinerkennung und beim Reinforcement Learning von Schlagparametern ein. Dadurch meistert der Roboter anspruchsvollere Spin- und Übungsszenarien als in vergleichbaren Arbeiten.Background: Robotic table tennis systems offer an ideal platform for pushing camera-based robotic manipulation systems to the limit. The unique challenge arises from the fast-paced play and the wide variation in spin and speed between strokes. The range of scenarios under which existing table tennis robots are able to operate is, however, limited, requiring slow play with low rotational velocity of the ball (spin). Research Goal: We aim to develop a table tennis robot system with learning capabilities able to handle spin against a human opponent. Methods: The robot system presented in this thesis consists of six components: ball position detection, ball spin detection, ball trajectory prediction, stroke parameter suggestion, robot trajectory generation, and robot control. For ball detection, the camera images pass through a conventional image processing pipeline. The ball’s 3D positions are determined using iterative triangulation and these are then used to estimate the current ball state (position and velocity). We propose three methods for estimating the spin. The first two methods estimate spin by analyzing the movement of the logo printed on the ball on high-resolution images using either conventional computer vision or convolutional neural networks. The final approach involves analyzing the trajectory of the ball using Magnus force fitting. Once the ball’s position, velocity, and spin are known, the future trajectory is predicted by forward-solving a physical ball model involving gravitational, drag, and Magnus forces. With the predicted ball state at hitting time as state input, we train a reinforcement learning algorithm to suggest the racket state at hitting time (stroke parameter). We use the Reflexxes library to generate a robot trajectory to achieve the suggested racket state. Results: Quantitative evaluation showed that all system components achieve results as good as or better than comparable robots. Regarding the research goal of this thesis, the robot was able to - maintain stable counter-hitting rallies of up to 60 balls with a human player, - return balls with different spin types (topspin and backspin) in the same rally, - learn multiple table tennis drills in just 200 strokes or fewer. Conclusion: Our spin detection system and reinforcement learning-based stroke parameter suggestion introduce significant algorithmic novelties. In contrast to previous work, our robot succeeds in more difficult spin scenarios and drills

Classification of table tennis strokes using a wearable device and deep learning

The analysis of sports using everyday mobile devices is an area that has been increasingly explored aiming to help the user to improve in all aspects of the sport. The objective of the work proposed for this dissertation is to developed application capable of detecting strokes in table tennis using the iPhone and the Apple Watch, in which a recorded table tennis strokes data set performed by several table tennis athletes was created to help develop the application. Since the Artificial Intelegence area is increasingly present in our daily lives, the motivation in this work is to have a first contact with the current state of AI, the technologies available and most used in today’s present, and as within the company, it was intended to begin research in this area, mainly using Apple devices, it was decided to try and create a mobile application capable of detecting strokes performed in table tennis that would work with devices capable of AI processing, in order to provide statistical data to help table tennis athletes and coaches, which can later be sell for. After a study of devices available on the apple market with the necessary capabilities for the purpose of the work, it was concluded that for this work, the devices to be used would be the iPhone (above the X model) and the Apple Watch (above the model 5). Also because there were no public table tennis data set available, a methodology was developed with the objective of capturing table tennis strokes trough motion data. The recording of motion data was done by using an application capable of recording sensors data using the apple watch who was used by each athlete on the wrist. The sensors used to record motion data were accelerometer and gyroscope, and the capture methodology was planned and overseen by coaches and athletes. From the methodology created, 2 base data sets were created. One consisting of a short interval between strokes and the second and last with a bigger interval between strokes. From these 2 data sets, 3 more were created with different pre processing configurations applied followed by a filtering and reformatting of data to the necessary format for the creation of a Deep Learning model. To generate a DL classifier model, two approaches were tested, one by using Create ML, and the other by using Convolution Neural Network-Long Short Term Memory and Convolution Neural Network-Long Short Term Memory architecture. To evaluate the models, statistics generated from training were saved during model testing and creation. Create ML data set classifier models showed average performance except in one data set, with the generated classifier model having a maximum performance of 89.66% F1 score while CNN-LSTM and ConvLSTM approach generated good performance from all data set generated classifier models with the best classifier being the ConvLSTM with a 97.33% F1 score. After the creation of this same model, development of the application was performed consisting of two parts, one on the iPhone where it is possible to see the statistics and another on the Apple Watch where the ML model is executed and the stroke performed is detected being then sent to the application on the iPhone. The final step consisted on evaluation of the application during a live game scenario followed by an user rating application feedback questionnaire on athletes and coaches. Final application feedback was positive across all subjects with recommendations to the application interface and improvements to the classifier model. The live game application scenario with the generated classifier model obtained a 80% correct labelled strokes

Parametric impact characterisation of a solid sports ball, WITH a view to developing a standard core for the GAA Sliotar

The main aim of this research was to characterise the dynamic impact behaviour of the sliotar core. Viscoelastic characterisation of the balls was conducted for a range of impact speeds. Modern polymer balls exhibited strain and strain-rate sensitivity while traditional multi-compositional balls exhibited strain dependency. The non-linear viscoelastic response was defined by two values of stiffness, initial and bulk stiffness. Traditional balls were up to 2.5 times stiffer than the modern types, with this magnitude being rate-dependent. The greater rate of increase of traditional ball stiffness produced a more non-linear COR velocity-dependence compared to modern balls. The dynamic stiffness results demonstrated limited applicability of quasi-static testing and springtheory equations. Analysis of ball deformation behaviour demonstrated that centre-of mass displacement and diameter compression values were not consistently equivalent for all ball types. The contribution of manufacturing conditions to ball performance was investigated by conducting extensive prototyping experiments. Manufacturing parameters of temperature, pressure and material composition were varied to produce a range of balls. Polymer hardness affected stiffness but not energy dissipation, with increased hardness increasing ball stiffness. The nucleating additive influenced ball COR, with increased additive tending to reduce ball COR, but this effect was sensitive to polymer grade. The impact response of the ball was simulated using three mathematical models. The first model was shown to replicate ball behaviour to only a limited degree, despite being used previously with reported success for other ball types. The second model exhibited a reasonable representation of ball impact response that was universally applicable to all tested ball types; however, the accuracy in terms of predicting force-displacement response was not as high as required for broad range implementation. The third model exhibited significantly better accuracy in simulating ball response. The force values generated from this model exhibited up to 95% agreement with experimental data

Camera calibration and configuration for estimation of tennis racket position in 3D.

Previously, stereo camera systems have been used to track markers attached to the racket frame, allowing for racket position to be measured in three-dimensions (3D). Typically, markers are manually selected on the image plane but this can be time consuming and inaccurate. The purpose of this project was to develop and validate a markerless method to estimate 3D racket position using a camera.The method relies on a silhouette of a racket captured with a camera whose relative pose (rotation and translation) is unknown. A candidate relative pose is used to measure the inconsistency between the silhouette and a set of racket silhouettes captured with a fully calibrated camera (known intrinsic and extrinsic parameters). The measure of inconsistency can be formulated as a cost function associated with the candidate relative pose. By adjusting parameters of the candidate relative pose to minimise the cost, an accurate estimation for the true 3D position of the racket can be made. Previous studies have found that silhouette-based pose optimisation methods depend on accurate camera calibration and silhouette extraction. Therefore, a repeatable and accurate camera calibration method to provide the relative pose of a camera with respect to a racket was developed. To facilitate silhouette extraction, the racket was painted black and a backlight was used.Synthetic camera poses and silhouette views associated with a 3D racket model were generated in Blender v2.70 and used to determine the optimum fully calibrated set configuration for a racket. A laboratory-based fully calibrated set (LFCS) consisting of 21 camera poses in a semispheric configuration was created. On average, using this set, racket position was reconstructed to within +/- 2 mm. This included systematic error arising from the calibration and error in the segmentation of silhouette boundaries. The maximum reconstruction error was 5.3 mm. Further synthetic testing demonstrated the methods ability to estimate 3D racket position during simulated real-play conditions. For racket silhouette orientations that simulated strokes expected to occur in tennis between 0 and 90°, mean RMSE for reconstruction of coordinates on the racket face plane was 1.5 +/- 1.8 mm. An RMSE of 2 mm was obtained from a camera positioned alongside the net, 14 m from the racket. Finally, this same camera position estimated 3D racket position to an accuracy of 1.9 +/- 0.14 mm using a fully calibrated set containing randomly orientated camera poses, during a simulated serve. This project developed and validated a novel markerless method to estimate 3D tennis racket position. A calibration method to obtain the relative pose of a camera with respect to a racket is presented and an appropriate configuration for a fully calibrated set is determined. The method has potential to be used alongside existing ball trajectory analysis tools to provide unprecedented information about player performance and to enhance tennis broadcasts. Future research should use the recommendations made in this project to inform and assist the development of the method for application during real tennis-play conditions