CLET: Computation of Latencies in Event-related potential Triggers using photodiode on virtual reality apparatuses

To investigate event-related activity in human brain dynamics as measured with EEG, triggers must be incorporated to indicate the onset of events in the experimental protocol. Such triggers allow for the extraction of ERP, i.e., systematic electrophysiological responses to internal or external stimuli that must be extracted from the ongoing oscillatory activity by averaging several trials containing similar events. Due to the technical setup with separate hardware sending and recording triggers, the recorded data commonly involves latency differences between the transmitted and received triggers. The computation of these latencies is critical for shifting the epochs with respect to the triggers sent. Otherwise, timing differences can lead to a misinterpretation of the resulting ERPs. This study presents a methodical approach for the CLET using a photodiode on a non-immersive VR (i.e., LED screen) and an immersive VR (i.e., HMD). Two sets of algorithms are proposed to analyze the photodiode data. The experiment designed for this study involved the synchronization of EEG, EMG, PPG, photodiode sensors, and ten 3D MoCap cameras with a VR presentation platform (Unity). The average latency computed for LED screen data for a set of white and black stimuli was 121.98 ± 8.71 ms and 121.66 ± 8.80 ms, respectively. In contrast, the average latency computed for HMD data for the white and black stimuli sets was 82.80 ± 7.63 ms and 69.82 ± 5.52 ms. The codes for CLET and analysis, along with datasets, tables, and a tutorial video for using the codes, have been made publicly available

Exergaming for Elderly: Subjective Experiences and Objective Movement Characteristics

Background: It is important for elderly to stay healthy and independent for as long as possible, and falls are a major cause for loss of independence. Physical activity aimed at improving balance that includes large movements and cognitive tasks has been shown to decrease fall risk. Using exergames as a training tool has increased in recent years, but the actual movements elicited by such exergames have yet to be investigated objectively. Aim: To investigate usability and enjoyment and provide objective quantification of movement size elicited by two exergames. Methods: Twenty healthy elderly (mean age 74.4, range 65-90) played two exergames, The Mole (SilverFit) and LightRace (YourShape: Fitness Evolved) at easy and medium level, with five trials of one minute at each level. Data on perceived exertion (BORG), enjoyment and system usability (SUS) was collected.. Movements were captured using OQUS Motion Capture System, with passive reflexive markers attached to the base of the 1st toe, heel and lumbar area of the back. Movement size was expressed as Interquartile range (IQR) of feet and trunk in all three directions, and as horizontal area covered by the lumbar and toe markers. Correlational analyses were performed to investigate relationships between game scores, BORG-scores, SUS-scores, IQR and area coverage. Repeated measures ANOVAs were used to analyze effects of game, level, and trial. Results: Both games scored high on usability, and the elderly perceived the games as enjoyable, relevant as physical activity, and not very exhausting. Game scores increased across trials and decreased from easy to medium levels. Nevertheless, participants preferred the medium over the easy levels because of the increased cognitive challenge. IQR and area in the feet exceeded those in the trunk, especially in the medio-lateral direction. There were no significant correlations between game score and movement variables. Discussion: The positive attitude from the participants is promising for future implementation of exergames into fall preventive exercises. However, the lack of correlations between game scores and movement variables indicate that although these exergames do not reward players for "cheating" movements; they have room for improvement concerning rewarding desired movements. Keywords: older adults, exergames, movement characteristics, stepping

Improving Exergame Technologies for Older Adults Using Machine Learning

Facilitating exercise for the aging population is an important focus area in the years to come. Exercise is one of the keys to healthy aging, and the most effective measure for preventing disease and loss of independence. Technology is already an important tool in healthy aging, and in recent years exercise games (exergames) have been shown to be a motivating, fun and efficient method of exercising. However, the existing technologies that facilitate use of exergames have some drawbacks that could decrease usability and accessibility of exergames for older adults. Advances in artificial intelligence have provided tools and methods that might be useful for improving exergame technologies, but it is not known how well these work in the context of balance exergames. The overall aim of this thesis is to explore how use of machine learning can improve existing solutions of core elements of exergaming systems used for balance training in elderly. Three research papers have been published as a result of the work in this thesis. These address three core aspects of exergame technologies: motion capture technology, movement pattern assessment, and force estimation. These papers together provide the following key findings: • A deep learning image analysis system is a viable option for accurately extracting joint center locations from digital video for use in in-home exergame settings. • A machine learning model can classify correctly performed medio-lateral weight-shifts in >9 out of 10 repetitions, without using pre-determined rules or thresholds. • Weight-shifting performance can be reliably estimated from joint center kinematic data using a recurrent neural network. In conclusion, we show that using machine learning models can make exergames more available and easy to use by eliminating possible barriers of use related to technological tools

Classification of movement quality in a weight-shifting exercise

In exercise games, it is often possible to gain rewards, i.e. points, by only partly completing an intended movement, which can undermine the effect of using such games for exercise. To ensure usability and reliability of exergames, correct movements must be accurately identified. Aim of the current study was to evaluate performance of machine learning models in classifying weightshifting movements as correct or incorrect. Eleven healthy elderly (6 F) performed a stepping exercise in a correct (with weight shift) and an incorrect (without weight shift) version. A 3D Motion Capture (3DMoCap) system calculated joint center positions (JCPs); 2270 repetitions (1133 correct) were recorded. Random Forest (RF), k-Nearest Neighbor (k-NN) and Support Vector Machine (SVM) classification models were built. Evaluation: 10fold leave-one-group-out cross validation (CV), repeated for all persons. Results showed high accuracy and recall in all classifiers. Average accuracy and recall was RF = 0.989, k-NN = 0.949, SVM = 0.958. Highest was RF on all JCPs, and SVM on shoulder JCPs (both 0.996). Lowest was k-NN on ankle JCPs (0.879). This study shows that all three models can distinguish correct and incorrect repetitions with high accuracy and recall, also by using selected JCPs. RF consistently outperformed the other models

Assessment of Machine Learning Models for Classification of Movement Patterns During a Weight-Shifting Exergame

In exercise gaming (exergaming), reward systems are typically based on rules/templates from joint movement patterns. These rules or templates need broad ranges in definitions of correct movement patterns to accommodate varying body shapes and sizes. This can lead to inaccurate rewards and, thus, inefficient exercise, which can be detrimental to progress. If exergames are to be used in serious settings like rehabilitation, accurate rewards for correctly performed movements are crucial. This article aims to investigate the level of accuracy machine learning/deep learning models can achieve in classification of correct repetitions naturally elicited from a weight-shifting exergame. Twelve healthy elderly (10F, age 70.4 SD 11.4) are recruited. Movements are captured using a marker-based 3-D motion-capture system. Random forest (RF), support vector machine, k-nearest neighbors, and multilayer perceptron (MLP) are the employed models, trained and tested on whole body movement patterns and on subsets of joints. MLP and RF reached the highest recall and F1-score, respectively, when using combined data from joint subsets. MLP recall range are 91% to 94%, and RF F1-score range 79% to 80%. MLP and RF also reached the highest recall and F1-score in each joint subset, respectively. Here, MLP ranged from 93% to 97% recall, while RF ranged from 73% to 80% F1-score. Recall results, show that >9 out of 10 repetitions are classified correctly, indicating that MLP/RF can be used to identify correctly performed repetitions of a weight-shifting exercise when using full-body data and when using joint subset data

Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training

Using standard digital cameras in combination with deep learning (DL) for pose estimation is promising for the in-home and independent use of exercise games (exergames). We need to investigate to what extent such DL-based systems can provide satisfying accuracy on exergame relevant measures. Our study assesses temporal variation (i.e., variability) in body segment lengths, while using a Deep Learning image processing tool (DeepLabCut, DLC) on two-dimensional (2D) video. This variability is then compared with a gold-standard, marker-based three-dimensional Motion Capturing system (3DMoCap, Qualisys AB), and a 3D RGB-depth camera system (Kinect V2, Microsoft Inc). Simultaneous data were collected from all three systems, while participants (N = 12) played a custom balance training exergame. The pose estimation DLC-model is pre-trained on a large-scale dataset (ImageNet) and optimized with context-specific pose annotated images. Wilcoxon’s signed-rank test was performed in order to assess the statistical significance of the differences in variability between systems. The results showed that the DLC method performs comparably to the Kinect and, in some segments, even to the 3DMoCap gold standard system with regard to variability. These results are promising for making exergames more accessible and easier to use, thereby increasing their availability for in-home exercise

Experiences of Stroke Survivors and Clinicians With a Fully Immersive Virtual Reality Treadmill Exergame for Stroke Rehabilitation: A Qualitative Pilot Study

Use of VR-games is considered a promising treatment approach in stroke rehabilitation. However, there is little knowledge on the use and expectations of patients and health professionals regarding the use of treadmill walking in a fully immersive virtual environment as a rehabilitation tool for gait training for stroke survivors. The objectives of the current study were to determine whether stroke survivors can use fully immersive VR utilizing modern HMDs while walking on a treadmill without adverse effects, and to investigate the experiences of stroke survivors and clinicians after testing with focus on acceptability and potential utilization in rehabilitation. A qualitative research design with semi-structured interviews was used to collect data. Five stroke survivors and five clinicians participated in the study and tested a custom-made VR-game on the treadmill before participating in individual semi-structured interview. Data were analyzed through thematic analysis. The analysis of the interview data identified two main categories: (1) experiencing acceptability through safety and motivation, and (2) implementing fully immersive VR in rehabilitation. Both stroke survivors' and clinicians enjoyed the treadmill-based VR-game and felt safe when using it. The stroke survivors experienced motivation for exercising and achievement by fulfilling tasks during the gaming session as the VR-game was engaging. The clinicians found additional motivation by competing in the game. Both groups saw a potential for use in gait rehabilitation after stroke, on the premise of individual adaptation to each patient's needs, and the technology being easy to use. The findings from this qualitative study suggest that a fully immersive treadmill-based VR-game is acceptable and potentially useful as part of gait rehabilitation after stroke, as it was positively received by both stroke survivors and clinicians working within stroke rehabilitation. The participants reported that they experienced motivation in the game through safety, engagement and achievement. They also saw the potential of implementing such a setup in their own rehabilitation setting. Elements that enable safety and engaging experience are important to maintain when using a fully immersive VR-game in stroke rehabilitation