Data analytics for image visual complexity and kinect-based videos of rehabilitation exercises

With the recent advances in computer vision and pattern recognition, methods from these fields are successfully applied to solve problems in various domains, including health care and social sciences. In this thesis, two such problems, from different domains, are discussed. First, an application of computer vision and broader pattern recognition in physical therapy is presented. Home-based physical therapy is an essential part of the recovery process in which the patient is prescribed specific exercises in order to improve symptoms and daily functioning of the body. However, poor adherence to the prescribed exercises is a common problem. In our work, we explore methods for improving home-based physical therapy experience. We begin by proposing DyAd, a dynamically difficulty adjustment system which captures the trajectory of the hand movement, evaluates the user's performance quantitatively and adjusts the difficulty level for the next trial of the exercise based on the performance measurements. Next, we introduce ExerciseCheck, a remote monitoring and evaluation platform for home-based physical therapy. ExerciseCheck is capable of capturing exercise information, evaluating the performance, providing therapeutic feedback to the patient and the therapist, checking the progress of the user over the course of the physical therapy, and supporting the patient throughout this period. In our experiments, Parkinson patients have tested our system at a clinic and in their homes during their physical therapy period. Our results suggests that ExerciseCheck is a user-friendly application and can assist patients by providing motivation, and guidance to ensure correct execution of the required exercises. As the second application, and within computer vision paradigm, we focus on visual complexity, an image attribute that humans can subjectively evaluate based on the level of details in the image. Visual complexity has been studied in psychophysics, cognitive science, and, more recently, computer vision, for the purposes of product design, web design, advertising, etc. We first introduce a diverse visual complexity dataset which compromises of seven image categories. We collect the ground-truth scores by comparing the pairwise relationship of images and then convert the pairwise scores to absolute scores using mathematical methods. Furthermore, we propose a method to measure the visual complexity that uses unsupervised information extraction from intermediate convolutional layers of deep neural networks. We derive an activation energy metric that combines convolutional layer activations to quantify visual complexity. The high correlations between ground-truth labels and computed energy scores in our experiments show superiority of our method compared to the previous works. Finally, as an example of the relationship between visual complexity and other image attributes, we demonstrate that, within the context of a category, visually more complex images are more memorable to human observers

RECOGNITION OF SPORTS EXERCISES USING INERTIAL SENSOR TECHNOLOGY

Supervised learning as a sub-discipline of machine learning enables the recognition of correlations between input variables (features) and associated outputs (classes) and the application of these to previously unknown data sets. In addition to typical areas of application such as speech and image recognition, fields of applications are also being developed in the sports and fitness sector. The purpose of this work was to implement a workflow for the automated recognition of sports exercises in the Matlab® programming environment and to carry out a comparison of different model structures. First, the acquisition of the sensor signals provided in the local network and their processing were implemented. The functionalities to be realised included the interpolation of lossy time series, the labelling of the activity intervals performed and, in part, the generation of sliding windows with statistical parameters. The preprocessed data were used for the training of classifiers and artificial neural networks (ANN). These were iteratively optimised in their corresponding hyper parameters for the data structure to be learned. The most reliable models were finally trained with an increased data set, validated and compared with regard to the achieved performance. In addition to the usual evaluation metrics such as F1 score and accuracy, the temporal behaviour of the assignments was also displayed graphically, which enabled statements to be made about potential causes for incorrect assignments. In this context, especially the transition areas between the classes were detected as erroneous assignments as well as exercises with insufficient or clearly deviating execution. The best overall accuracy achieved with ANN and the increased dataset was 93.7 %

IMU-based classification of resistive exercises for real-time training monitoring on board the international space station with potential telemedicine spin-off

The microgravity exposure that astronauts undergo during space missions lasting up to 6 months induces biochemical and physiological changes potentially impacting on their health. As a countermeasure, astronauts perform an in-flight training program consisting in different resistive exercises. To train optimally and safely, astronauts need guidance by on-ground specialists via a real-time audio/video system that, however, is subject to a communication delay that increases in proportion to the distance between sender and receiver. The aim of this work was to develop and validate a wearable IMU-based biofeedback system to monitor astronauts in-flight training displaying real-time feedback on exercises execution. Such a system has potential spin-offs also on personalized home/remote training for fitness and rehabilitation. 29 subjects were recruited according to their physical shape and performance criteria to collect kinematics data under ethical committee approval. Tests were conducted to (i) compare the signals acquired with our system to those obtained with the current state-of-the-art inertial sensors and (ii) to assess the exercises classification performance. The magnitude square coherence between the signals collected with the two different systems shows good agreement between the data. Multiple classification algorithms were tested and the best accuracy was obtained using a MultiLayer Perceptron (MLP). MLP was also able to identify mixed errors during the exercise execution, a scenario that is quite common during training. The resulting system represents a novel low-cost training monitor tool that has space application, but also potential use on Earth for individuals working-out at home or remotely thanks to its ease of use and portability

Recognition and repetition counting for local muscular endurance exercises in exercise-based rehabilitation: a comparative study using artificial Intelligence models

Exercise-based cardiac rehabilitation requires patients to perform a set of certain prescribed exercises a specific number of times. Local muscular endurance exercises are an important part of the rehabilitation program. Automatic exercise recognition and repetition counting, from wearable sensor data, is an important technology to enable patients to perform exercises independently in remote settings, e.g. their own home. In this paper, we first report on a comparison of traditional approaches to exercise recognition and repetition counting (supervised ML and peak detection) with Convolutional Neural Networks (CNNs). We investigated CNN models based on the AlexNet architecture and found that the performance was better than the traditional approaches, for exercise recognition (overall F1-score of 97.18%) and repetition counting (±1 error among 90% observed sets). To the best of our knowledge, our approach of using a single CNN method for both recognition and repetition counting is novel. Also, we make the INSIGHT-LME dataset publicly available to encourage further research

TEACHING MUSIC LITERACY IN BEGINNING BAND: A REVIEW OF METHOD BOOKS AND A PROPOSED ALTERNATIVE METHOD

The purpose of this study is to review current band methodologies, specifically for the beginning band, and identify the current process of teaching music literacy. For beginning bands, music literacy instruction is a powerful tool music students use to be successful at sight-reading. It cannot be assumed that simply because students perform music on instruments, they also will achieve high ratings at sight-reading. Brain-compatible learning may offer a solution in addressing the issue with how beginning band students are taught to read music. Therefore, this study will propose a more effective method, applying the Music Learning Theory of Edwin Gordon to teaching music literacy using brain-compatible teaching strategies

Performing Realistic Workout Activity Recognition on Consumer Smartphones

Smartphones have become an essential part of our lives. Especially its computing power and its current specifications make a modern smartphone a powerful device for human activity recognition tasks. Equipped with various integrated sensors, a modern smartphone can be leveraged for lots of smart applications. We already investigated the possibility of using an unmodified commercial smartphone to recognize eight strength-based exercises. App-based workouts have become popular in the last few years. The advantage of using a mobile device is that you can practice anywhere at anytime. In our previous work, we proved the possibility of turning a commercial smartphone into an active sonar device to leverage the echo reflected from exercising movement close to the device. By conducting a test study with 14 participants, we showed the first results for cross person evaluation and the generalization ability of our inference models on disjoint participants. In this work, we extended another model to further improve the model generalizability and provided a thorough comparison of our proposed system to other existing state-of-the-art approaches. Finally, a concept of counting the repetitions is also provided in this study as a parallel task to classification