Smart Wearables for Tennis Game Performance Analysis

For monitoring the progress of athletes in various sports and disciplines, several different approaches are nowadays available. Recently, miniature wearables have gained popularity for this task due to being lightweight and typically cheaper than other approaches. They can be positioned on the athlete’s body, or in some cases, the devices are incorporated into sports requisites, like tennis racquet handles, balls, baseball bats, gloves, etc. Their purpose is to monitor the performance of an athlete by gathering essential information during match or training. In this chapter, the focus will be on the different possibilities of tennis game monitoring analysis. A miniature wearable device, which is worn on a player’s wrist during the activity, is going to be presented and described. The smart wearable device monitors athletes’ arm movements with sampling the output of the 6 DOF IMU. Parallel to that, it also gathers biometric information like pulse rate and skin temperature. All the collected information is stored locally on the device during the sports activity. Later, it can be downloaded to a PC and transferred to a cloud-based service, where visualization of the recorded data and more detailed game/training statistics can be performed

A framework for comprehensive analysis of a swing in sports using low-cost inertial sensors

We present a novel framework to monitor the three- dimensional trajectory (orientation and position) of a golf swing using miniaturized inertial sensors. Firstly we employed a highly accurate and computationally efficient revised gradient descent algorithm to obtain the orientation of a golf club. Secondly, we designed a series of digital filters to determine the backward and forward segments of the swing, enabling us to calculate drift-free linear velocity along with the relative 3D position of the golf club during the entire swing. Finally, the calculated motion trajectory was verified against a ground truth VICON system using Iterative Closest Point (ICP) in conjunction with Principal Component Analysis (PCA). The computationally efficient framework present here achieves a high level of accuracy (r = 0.9885, p < 0.0001) for such a low-cost system. This framework can be utilized for reliable movement technique evaluation and can provide near real-time feedback for athletes in various unconstrained environments. It is envisaged that the proposed framework is applicable to other racket based sports (e.g. tennis, cricket and hurling)

Artificial Intelligence Of Things For Ubiquitous Sports Analytics

To enable mobile devices to perform in-the-wild sports analytics, particularly swing tracking, remains an open question. A crucial challenge is to develop robust methods that can operate across various sports (e.g., golf and tennis), different sensors (cameras and IMU), and diverse human users. Traditional approaches typically rely on vision-based or IMU-based methods to extract key points from subjects in order to estimate trajectory predictions. However, these methods struggle to generate accurate swing tracking, as vision-based techniques are susceptible to occlusion, and IMU sensors are notorious for accumulated errors. In this thesis, we propose several innovative solutions by leveraging AIoT, including the IoT with ubiquitous wearable devices such as smartphones and smart wristbands, and harnessing the power of AI such as deep neural networks, to achieve ubiquitous sports analytics. We make three main technical contributions: a tailored deep neural network design, network model automatic search, and model domain adaptation to address the problem of heterogeneity among devices, human subjects, and sports for ubiquitous sports analytics. In Chapter 2, we begin with the design of a prototype that combines IMU and depth sensor fusion, along with a tailored deep neural network, to address the occlusion problems faced by depth sensors during swings. To recover swing trajectories with fine-grained details, we propose a CNN-LSTM architecture that learns multi-modalities within depth and IMU sensor fusion. In Chapter 3, we develop a framework to reduce the overhead of model design for new devices, sports, and human users. By designing a regression-based stochastic NAS method, we improve swing-tracking algorithms through automatic model generation. We also extend our studies to include unseen human users, sensor devices, and sports. Leveraging a domain adaptation method, we propose a framework that eliminates the need for tedious training data collection and labeling for new users, devices, and sports via adversarial learning. In Chapter 4, we present a framework to alleviate the model parameter selection process in NAS, as introduced in Chapter 3. By employing zero-cost proxies, we search for the optimal swing tracking architecture without training, in a significantly larger candidate model pool. We demonstrate that the proposed method outperforms state-of-the-art approaches in swing tracking, as well as in adapting to different subjects, sports, and devices. Overall, this thesis develops a series of innovative machine learning algorithms to enable ubiquitous IoT wearable devices to perform accurate swing analytics (e.g., tracking, analysis, and assessment) in real-world conditions

Wearables for Movement Analysis in Healthcare

Quantitative movement analysis is widely used in clinical practice and research to investigate movement disorders objectively and in a complete way. Conventionally, body segment kinematic and kinetic parameters are measured in gait laboratories using marker-based optoelectronic systems, force plates, and electromyographic systems. Although movement analyses are considered accurate, the availability of specific laboratories, high costs, and dependency on trained users sometimes limit its use in clinical practice. A variety of compact wearable sensors are available today and have allowed researchers and clinicians to pursue applications in which individuals are monitored in their homes and in community settings within different fields of study, such movement analysis. Wearable sensors may thus contribute to the implementation of quantitative movement analyses even during out-patient use to reduce evaluation times and to provide objective, quantifiable data on the patients’ capabilities, unobtrusively and continuously, for clinical purposes

Recent Advances in Motion Analysis

The advances in the technology and methodology for human movement capture and analysis over the last decade have been remarkable. Besides acknowledged approaches for kinematic, dynamic, and electromyographic (EMG) analysis carried out in the laboratory, more recently developed devices, such as wearables, inertial measurement units, ambient sensors, and cameras or depth sensors, have been adopted on a wide scale. Furthermore, computational intelligence (CI) methods, such as artificial neural networks, have recently emerged as promising tools for the development and application of intelligent systems in motion analysis. Thus, the synergy of classic instrumentation and novel smart devices and techniques has created unique capabilities in the continuous monitoring of motor behaviors in different fields, such as clinics, sports, and ergonomics. However, real-time sensing, signal processing, human activity recognition, and characterization and interpretation of motion metrics and behaviors from sensor data still representing a challenging problem not only in laboratories but also at home and in the community. This book addresses open research issues related to the improvement of classic approaches and the development of novel technologies and techniques in the domain of motion analysis in all the various fields of application

Moving On:Measuring Movement Remotely after Stroke

Most persons with stroke suffer from motor impairment, which restricts mobility on one side, and affects their independence in daily life activities. Measuring recovery is needed to develop individualized therapies. However, commonly used clinical outcomes suffer from low resolution and subjectivity. Therefore, objective biomechanical metrics should be identified to measure movement quality. However, non-portable laboratory setups are required in order to measure these metrics accurately. Alternatively, minimal wearable systems can be developed to simplify measurements performed at clinic or home to monitor recovery. Thus, the goal of the thesis was ‘To identify metrics that reflect movement quality of upper and lower extremities after stroke and develop wearable minimal systems for tracking the proposed metrics’. Section Upper Extremity First, we systematically reviewed literature ( Chapter II ) to identify metrics used to measure reaching recovery longitudinally post-stroke. Although several metrics were found, it was not clear how they differentiated recovery from compensation strategies. Future studies must address this gap in order to optimize stroke therapy. Next, we assessed a ‘valid’ measure for smoothness of upper paretic limb reaching ( Chapter III ), as this was commonly used to measure movement quality. After a systematic review and simulation analyses, we found that reaching smoothness is best measured using spectral arc length. The studies in this section offer us a better understanding of movement recovery in the upper extremity post-stroke. Section Lower Extremity Although metrics that reflect gait recovery are yet to be identified, in this section we focused on developing minimal solutions to measure gait quality. First, we showed the feasibility of 1D pressure insoles as a lightweight alternative for measuring 3D Ground Reaction Forces (GRF) ( Chapter IV ). In the following chapters, we developed a minimal system; the Portable Gait Lab (PGL) using only three Inertial Measurement Units (IMUs) (one per foot and one on the pelvis). We explored the Centroidal Moment Pivot (CMP) point ( Chapter V ) as a biomechanical constraint that can help with the reduction in sensors. Then, we showed the feasibility of the PGL to track 3D GRF ( Chapters VI-VII ) and relative foot and CoM kinematics ( Chapter VIII-IX ) during variable overground walking by healthy participants. Finally, we performed a limited validation study in persons with chronic stroke ( Chapter X ). This thesis offers knowledge and tools which can help clinicians and researchers understand movement quality and thereby develop individualized therapies post-stroke

Trampling Analysis of Autonomous Mowers: Implications on Garden Designs

Several trials have been carried out by various authors concerning autonomous mowers, which are battery-powered machines. The effects of these machines on turfgrass quality and energy consumption have been thoroughly investigated. However, there are still some aspects that have not been studied. Among these, random trajectory overlapping is one of the most important. To investigate these aspects, two RTK-GPS devices along with the custom-built software used for previous trials has been upgraded in order to precisely calculate how many times the mower drives over the same spot using random trajectories. This parameter, the number of passages in the same position, was hypothesized to explain the autonomous mower's overlapping and trampling action. The trial has been carried out testing a commercial autonomous mower on three areas with different levels of complexity to assess its performances. The following variables were examined: the percentage of mowed area, the distance travelled, the number of intersections, the number of passages, and the autonomous mower's work efficiency. The average percentage of area mown (average value for the three areas) was 54.64% after one hour and 80.15% after two hours of work. Percentage of area mown was 15% higher for the area with no obstacles after two hours of work. The number of passages was slightly different among the three garden designs. The garden with no obstacles obtained the highest number of passages with an average of 37 passages. The highest working efficiency was obtained in the garden with an intermediate number of obstacles with a value of 0.40 after two hours of work. The estimated energy consumption resulted 0.31 Wh m(-2) after one hour and 0.42 Wh m(-2) after two hours of working. These results highlight how the correct settings of cutting time may be crucial to consistently save energy during the long period and may be useful for a complete automation of the maintenance of green areas

Human action recognition and mobility assessment in smart environments with RGB-D sensors

openQuesta attività di ricerca è focalizzata sullo sviluppo di algoritmi e soluzioni per ambienti intelligenti sfruttando sensori RGB e di profondità. In particolare, gli argomenti affrontati fanno riferimento alla valutazione della mobilità di un soggetto e al riconoscimento di azioni umane. Riguardo il primo tema, l'obiettivo è quello di implementare algoritmi per l'estrazione di parametri oggettivi che possano supportare la valutazione di test di mobilità svolta da personale sanitario. Il primo algoritmo proposto riguarda l'estrazione di sei joints sul piano sagittale utilizzando i dati di profondità forniti dal sensore Kinect. La precisione in termini di stima degli angoli di busto e ginocchio nella fase di sit-to-stand viene valutata considerando come riferimento un sistema stereofotogrammetrico basato su marker. Un secondo algoritmo viene proposto per facilitare la realizzazione del test in ambiente domestico e per consentire l'estrazione di un maggior numero di parametri dall'esecuzione del test Timed Up and Go. I dati di Kinect vengono combinati con quelli di un accelerometro attraverso un algoritmo di sincronizzazione, costituendo un setup che può essere utilizzato anche per altre applicazioni che possono beneficiare dell'utilizzo congiunto di dati RGB, profondità ed inerziali. Vengono quindi proposti algoritmi di rilevazione della caduta che sfruttano la stessa configurazione del Timed Up and Go test. Per quanto riguarda il secondo argomento affrontato, l'obiettivo è quello di effettuare la classificazione di azioni che possono essere compiute dalla persona all'interno di un ambiente domestico. Vengono quindi proposti due algoritmi di riconoscimento attività i quali utilizzano i joints dello scheletro di Kinect e sfruttano un SVM multiclasse per il riconoscimento di azioni appartenenti a dataset pubblicamente disponibili, raggiungendo risultati confrontabili con lo stato dell'arte rispetto ai dataset CAD-60, KARD, MSR Action3D.This research activity is focused on the development of algorithms and solutions for smart environments exploiting RGB and depth sensors. In particular, the addressed topics refer to mobility assessment of a subject and to human action recognition. Regarding the first topic, the goal is to implement algorithms for the extraction of objective parameters that can support the assessment of mobility tests performed by healthcare staff. The first proposed algorithm regards the extraction of six joints on the sagittal plane using depth data provided by Kinect sensor. The accuracy in terms of estimation of torso and knee angles in the sit-to-stand phase is evaluated considering a marker-based stereometric system as a reference. A second algorithm is proposed to simplify the test implementation in home environment and to allow the extraction of a greater number of parameters from the execution of the Timed Up and Go test. Kinect data are combined with those of an accelerometer through a synchronization algorithm constituting a setup that can be used also for other applications that benefit from the joint usage of RGB, depth and inertial data. Fall detection algorithms exploiting the same configuration of the Timed Up and Go test are therefore proposed. Regarding the second topic addressed, the goal is to perform the classification of human actions that can be carried out in home environment. Two algorithms for human action recognition are therefore proposed, which exploit skeleton joints of Kinect and a multi-class SVM for the recognition of actions belonging to publicly available datasets, achieving results comparable with the state of the art in the datasets CAD-60, KARD, MSR Action3D.INGEGNERIA DELL'INFORMAZIONECippitelli, EneaCippitelli, Ene

Human action recognition and mobility assessment in smart environments with RGB-D sensors

Questa attività di ricerca è focalizzata sullo sviluppo di algoritmi e soluzioni per ambienti intelligenti sfruttando sensori RGB e di profondità. In particolare, gli argomenti affrontati fanno riferimento alla valutazione della mobilità di un soggetto e al riconoscimento di azioni umane. Riguardo il primo tema, l'obiettivo è quello di implementare algoritmi per l'estrazione di parametri oggettivi che possano supportare la valutazione di test di mobilità svolta da personale sanitario. Il primo algoritmo proposto riguarda l'estrazione di sei joints sul piano sagittale utilizzando i dati di profondità forniti dal sensore Kinect. La precisione in termini di stima degli angoli di busto e ginocchio nella fase di sit-to-stand viene valutata considerando come riferimento un sistema stereofotogrammetrico basato su marker. Un secondo algoritmo viene proposto per facilitare la realizzazione del test in ambiente domestico e per consentire l'estrazione di un maggior numero di parametri dall'esecuzione del test Timed Up and Go. I dati di Kinect vengono combinati con quelli di un accelerometro attraverso un algoritmo di sincronizzazione, costituendo un setup che può essere utilizzato anche per altre applicazioni che possono beneficiare dell'utilizzo congiunto di dati RGB, profondità ed inerziali. Vengono quindi proposti algoritmi di rilevazione della caduta che sfruttano la stessa configurazione del Timed Up and Go test. Per quanto riguarda il secondo argomento affrontato, l'obiettivo è quello di effettuare la classificazione di azioni che possono essere compiute dalla persona all'interno di un ambiente domestico. Vengono quindi proposti due algoritmi di riconoscimento attività i quali utilizzano i joints dello scheletro di Kinect e sfruttano un SVM multiclasse per il riconoscimento di azioni appartenenti a dataset pubblicamente disponibili, raggiungendo risultati confrontabili con lo stato dell'arte rispetto ai dataset CAD-60, KARD, MSR Action3D.This research activity is focused on the development of algorithms and solutions for smart environments exploiting RGB and depth sensors. In particular, the addressed topics refer to mobility assessment of a subject and to human action recognition. Regarding the first topic, the goal is to implement algorithms for the extraction of objective parameters that can support the assessment of mobility tests performed by healthcare staff. The first proposed algorithm regards the extraction of six joints on the sagittal plane using depth data provided by Kinect sensor. The accuracy in terms of estimation of torso and knee angles in the sit-to-stand phase is evaluated considering a marker-based stereometric system as a reference. A second algorithm is proposed to simplify the test implementation in home environment and to allow the extraction of a greater number of parameters from the execution of the Timed Up and Go test. Kinect data are combined with those of an accelerometer through a synchronization algorithm constituting a setup that can be used also for other applications that benefit from the joint usage of RGB, depth and inertial data. Fall detection algorithms exploiting the same configuration of the Timed Up and Go test are therefore proposed. Regarding the second topic addressed, the goal is to perform the classification of human actions that can be carried out in home environment. Two algorithms for human action recognition are therefore proposed, which exploit skeleton joints of Kinect and a multi-class SVM for the recognition of actions belonging to publicly available datasets, achieving results comparable with the state of the art in the datasets CAD-60, KARD, MSR Action3D

A Reconfigurable and wearable wireless sensor system and its case study in the training of hammer throwers

Wearable sensors have been popularly used in many applications with the development of computer science and engineering. However, wearables for biomechanical feedback in motor learning and training are still rare. Therefore, this thesis focuses on developing an efficient and cost-effective wireless sensor system through a case study on the hammer throw. The results have shown that the proposed reconfigurable and wearable system can implement real-time biomechanical feedback in the hammer-throw training. Furthermore, the experimental results suggest that various throw-control patterns could be identified by using one tension-sensor and two inertial measurement units (i.e., more superior practicality than 3D motion capture), indicating that the low-cost wearable system has potential to substitute the expensive 3D motion capture technology. The proposed system can be easily modified and applied to many other applications, including but not limited to healthcare, rehabilitation, and smart homes, etc.National Science and Engineering Research Council of Canada (NSERC