Hockey Pose Estimation and Action Recognition using Convolutional Neural Networks to Ice Hockey

Human pose estimation and action recognition in ice hockey are one of the biggest challenges in computer vision-driven sports analytics, with a variety of difficulties such as bulky hockey wear, color similarity between ice rink and player jersey and the presence of additional sports equipment used by the players such as hockey sticks. As such, deep neural network architectures typically used for sports including baseball, soccer, and track and field perform poorly when applied to hockey. This research involves the design and implementation of deep neural networks for both pose estimation and action recognition can effectively evaluate the pose and the actions of a hockey player. First, a pre-trained convolutional neural network, known as the stacked hourglass network, is used to determine a hockey player's body placement in video frames, also known as pose estimation. The proposed method provides a tool to analyze the pose of a hockey player via broadcast video which aids in the eventual assessment of a hockey player's speed, shot accuracy, or other metrics. The algorithm demonstrated to be successful since it identifies on average 81.56% of the joints of a hockey player on a set of test images. Furthermore, inspired by the idea that modeling the pose of a hockey stick can improve hockey player pose estimation, a novel deep learning computer vision architecture known as the HyperStackNet has been designed and implemented for joint player and stick pose estimation. In addition to improving player pose estimation, the HyperStackNet architecture enables improved transfer learning from pre-trained stacked hourglass networks trained on a different domain. Experimental results demonstrate that when the HyperStackNet is trained to detect 18 different joint positions on a hockey player (including the hockey stick), the accuracy is 98.8% on the test dataset, thus demonstrating its efficacy for handling complex joint player and stick pose estimation from video. Extending from pose recognition, this research involves the development of an algorithm for accurate recognition of actions for hockey. To perform this action recognition, a convolutional neural network estimates actions through unifying latent pose and action recognition. The action recognition hourglass network, or ARHN, is designed to interpret player actions in ice hockey video using estimated pose. ARHN has three components. The first component is the latent pose estimator, the second transforms latent features to a common frame of reference, and the third performs action recognition. Since no benchmark dataset for pose estimation or action recognition is available for hockey players, we first had to generate such an annotated dataset. Experimental results show action recognition accuracy of 65% for four types of actions in hockey. When similar poses are merged to three and two classes, the accuracy rate increases to 71% and 78%, proving the potential of the methodology for automated action recognition in hockey

Data-guided statistical sparse measurements modeling for compressive sensing

Digital image acquisition can be a time consuming process for situations where high spatial resolution is required. As such, optimizing the acquisition mechanism is of high importance for many measurement applications. Acquiring such data through a dynamically small subset of measurement locations can address this problem. In such a case, the measured information can be regarded as incomplete, which necessitates the application of special reconstruction tools to recover the original data set. The reconstruction can be performed based on the concept of sparse signal representation. Recovering signals and images from their sub-Nyquist measurements forms the core idea of compressive sensing (CS). In this work, a CS-based data-guided statistical sparse measurements method is presented, implemented and evaluated. This method significantly improves image reconstruction from sparse measurements. In the data-guided statistical sparse measurements approach, signal sampling distribution is optimized for improving image reconstruction performance. The sampling distribution is based on underlying data rather than the commonly used uniform random distribution. The optimal sampling pattern probability is accomplished by learning process through two methods - direct and indirect. The direct method is implemented for learning a nonparametric probability density function directly from the dataset. The indirect learning method is implemented for cases where a mapping between extracted features and the probability density function is required. The unified model is implemented for different representation domains, including frequency domain and spatial domain. Experiments were performed for multiple applications such as optical coherence tomography, bridge structure vibration, robotic vision, 3D laser range measurements and fluorescence microscopy. Results show that the data-guided statistical sparse measurements method significantly outperforms the conventional CS reconstruction performance. Data-guided statistical sparse measurements method achieves much higher reconstruction signal-to-noise ratio for the same compression rate as the conventional CS. Alternatively, Data-guided statistical sparse measurements method achieves similar reconstruction signal-to-noise ratio as the conventional CS with significantly fewer samples

Optical Microsystems for Static and Dynamic Tactile Sensing: Design, Modeling, Fabrication and Testing

Minimally invasive surgical operations encompass various surgical tasks ranging from conventional endoscopic/laparoscopic methods to recent sophisticated minimally invasive surgical techniques. In such sophisticated techniques, surgeons use equipment varying from robotic-assisted surgical platforms for abdominal surgery to computer-controlled catheters for catheter-based cardiovascular surgery. Presently, the countless advantages that minimally invasive surgery offers for both patients and surgeons have made the use of such surgical operations routine and reliable. However, in such operations, unlike conventional surgical operations, surgeons still suffer from the lack of tactile perception while interacting with the biological tissues using surgical instruments. To address this issue, it is necessary to develop a tactile sensor that can mimic the fingertip tactile perceptions of surgeons. In doing so and to satisfy the needs of surgeons, a number of considerations should be implemented in the design of the tactile sensors. First, the sensor should be magnetic resonance compatible to perform measurements even in the presence of magnetic resonance imaging (MRI) devices. Currently, such devices are in wide-spread use in surgical operation rooms. Second, the sensor should be electrically-passive because introducing electrical current into the patients’ body is not desirable in various surgical operations such as cardiovascular operations. Third, the sensor should perform measurements under both static and dynamic loading conditions during the sensor-tissue interactions. Such a capability of the sensor ensures that surgeons receive tactile feedback even when there is continuous static contact between surgical tools and tissues. Essentially, surgeons need such feedback to make surgical tasks safer. In addition, the size of the sensor should be miniaturized to address the size restrictions. In fact, the combination of intensity-based optical fiber sensing principles and micro-systems technology is one of the limited choices that address all the required considerations to develop such tactile sensors in a variety of ways. The present thesis deals with the design, modeling, manufacturing, testing, and characterizing of different tactile sensor configurations based on detection and integration methods. The various stages of design progress and principles are developed into different design configurations and presented in different chapters. The main sensing principle applied is based on the intensity modulation principle of optical fibers using micro-systems technology. In addition, a hybrid sensing principle is also studied by integrating both optical and non-optical detection methods. The micromachined sensors are categorized into five different generations. Each generation has advantages by comparison with its counterpart from the previous generation. The initial development of micromachined sensors is based on optical fiber coupling loss. In the second phase, a hybrid optical-piezoresistive sensing principle is studied. The success of these phases was instrumental in realizing a micromachined sensor that has the advantage of being fully optical. This sensor measures the magnitude of concentrated and distributed force, the position of a concentrated force, the variations in the force distribution along its length, the relative hardness of soft contact objects, and the local discontinuities in the hardness of the contact objects along the length of the contact area. Unlike most electrical-based commercially-available sensors, it performs all of these measurements under both static and dynamic loading conditions. Moreover, it is electrically passive and potentially MRI-compatible. The performances of the sensors were experimentally characterized for specific conditions presented in this thesis. However, these performances are easily tunable and adjustable depending upon the requirements of specific surgical tasks. Although the sensors were initially designed for surgical applications, they can have numerous other applications in the areas of robotics, automation, tele-display, and material testing

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing

Fundamental Approaches to Software Engineering

This open access book constitutes the proceedings of the 24th International Conference on Fundamental Approaches to Software Engineering, FASE 2021, which took place during March 27–April 1, 2021, and was held as part of the Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg but changed to an online format due to the COVID-19 pandemic. The 16 full papers presented in this volume were carefully reviewed and selected from 52 submissions. The book also contains 4 Test-Comp contributions

XX Workshop de Investigadores en Ciencias de la Computación - WICC 2018 : Libro de actas

Actas del XX Workshop de Investigadores en Ciencias de la Computación (WICC 2018), realizado en Facultad de Ciencias Exactas y Naturales y Agrimensura de la Universidad Nacional del Nordeste, los dìas 26 y 27 de abril de 2018.Red de Universidades con Carreras en Informática (RedUNCI

XX Workshop de Investigadores en Ciencias de la Computación - WICC 2018 : Libro de actas

Actas del XX Workshop de Investigadores en Ciencias de la Computación (WICC 2018), realizado en Facultad de Ciencias Exactas y Naturales y Agrimensura de la Universidad Nacional del Nordeste, los dìas 26 y 27 de abril de 2018.Red de Universidades con Carreras en Informática (RedUNCI