Artificial Intelligence Of Things For Ubiquitous Sports Analytics

To enable mobile devices to perform in-the-wild sports analytics, particularly swing tracking, remains an open question. A crucial challenge is to develop robust methods that can operate across various sports (e.g., golf and tennis), different sensors (cameras and IMU), and diverse human users. Traditional approaches typically rely on vision-based or IMU-based methods to extract key points from subjects in order to estimate trajectory predictions. However, these methods struggle to generate accurate swing tracking, as vision-based techniques are susceptible to occlusion, and IMU sensors are notorious for accumulated errors. In this thesis, we propose several innovative solutions by leveraging AIoT, including the IoT with ubiquitous wearable devices such as smartphones and smart wristbands, and harnessing the power of AI such as deep neural networks, to achieve ubiquitous sports analytics. We make three main technical contributions: a tailored deep neural network design, network model automatic search, and model domain adaptation to address the problem of heterogeneity among devices, human subjects, and sports for ubiquitous sports analytics. In Chapter 2, we begin with the design of a prototype that combines IMU and depth sensor fusion, along with a tailored deep neural network, to address the occlusion problems faced by depth sensors during swings. To recover swing trajectories with fine-grained details, we propose a CNN-LSTM architecture that learns multi-modalities within depth and IMU sensor fusion. In Chapter 3, we develop a framework to reduce the overhead of model design for new devices, sports, and human users. By designing a regression-based stochastic NAS method, we improve swing-tracking algorithms through automatic model generation. We also extend our studies to include unseen human users, sensor devices, and sports. Leveraging a domain adaptation method, we propose a framework that eliminates the need for tedious training data collection and labeling for new users, devices, and sports via adversarial learning. In Chapter 4, we present a framework to alleviate the model parameter selection process in NAS, as introduced in Chapter 3. By employing zero-cost proxies, we search for the optimal swing tracking architecture without training, in a significantly larger candidate model pool. We demonstrate that the proposed method outperforms state-of-the-art approaches in swing tracking, as well as in adapting to different subjects, sports, and devices. Overall, this thesis develops a series of innovative machine learning algorithms to enable ubiquitous IoT wearable devices to perform accurate swing analytics (e.g., tracking, analysis, and assessment) in real-world conditions