A Person-Centric Design Framework for At-Home Motor Learning in Serious Games

abstract: In motor learning, real-time multi-modal feedback is a critical element in guided training. Serious games have been introduced as a platform for at-home motor training due to their highly interactive and multi-modal nature. This dissertation explores the design of a multimodal environment for at-home training in which an autonomous system observes and guides the user in the place of a live trainer, providing real-time assessment, feedback and difficulty adaptation as the subject masters a motor skill. After an in-depth review of the latest solutions in this field, this dissertation proposes a person-centric approach to the design of this environment, in contrast to the standard techniques implemented in related work, to address many of the limitations of these approaches. The unique advantages and restrictions of this approach are presented in the form of a case study in which a system entitled the "Autonomous Training Assistant" consisting of both hardware and software for guided at-home motor learning is designed and adapted for a specific individual and trainer. In this work, the design of an autonomous motor learning environment is approached from three areas: motor assessment, multimodal feedback, and serious game design. For motor assessment, a 3-dimensional assessment framework is proposed which comprises of 2 spatial (posture, progression) and 1 temporal (pacing) domains of real-time motor assessment. For multimodal feedback, a rod-shaped device called the "Intelligent Stick" is combined with an audio-visual interface to provide feedback to the subject in three domains (audio, visual, haptic). Feedback domains are mapped to modalities and feedback is provided whenever the user's performance deviates from the ideal performance level by an adaptive threshold. Approaches for multi-modal integration and feedback fading are discussed. Finally, a novel approach for stealth adaptation in serious game design is presented. This approach allows serious games to incorporate motor tasks in a more natural way, facilitating self-assessment by the subject. An evaluation of three different stealth adaptation approaches are presented and evaluated using the flow-state ratio metric. The dissertation concludes with directions for future work in the integration of stealth adaptation techniques across the field of exergames.Dissertation/ThesisDoctoral Dissertation Computer Science 201

Fatigue effect on task performance in haptic virtual environment for home-based rehabilitation

Stroke rehabilitation is to train the motor function of a patient’s limb. In this process, functional assessment is of importance, and it is primarily based on a patient’s task performance. The context of the rehabilitation discussed in this thesis is such that functional assessment is conducted through a computer system and the Internet. In particular, a patient performs the task at home in a haptic virtual environment, and the task performance is transmitted to the therapist over the Internet. One problem with this approach to functional assessment is that a patient’s mind state is little known to the therapist. This immediately leads to one question, that is, whether an elevated mind state will have some significant effect on the patient’s task performance? If so, this approach can result in a considerable error. The overall objective of this thesis study was to generate an answer to the aforementioned question. The study focused on a patient’s elevated fatigue state. The specific objectives of the study include: (i) developing a haptic virtual environment prototype system for functional assessment, (ii) developing a physiological-based inference system for fatigue state, and (iii) performing an experiment to generate knowledge regarding the fatigue effect on task performance. With a limited resource in recruiting patients in the experiment, the study conducted few experiments on patients but mostly on healthy subjects. The study has concluded: (1) the proposed haptic virtual environment system is effective for the wrist coordination task and is likely promising to other tasks, (2) the accuracy of proposed fatigue inference system achieves 89.54%, for two levels of fatigue state, which is promising, (3) the elevated fatigue state significantly affects task performance in the context of wrist coordination task, and (4) the accuracy of the individual-based inference approach is significantly higher than that of the group-based inference approach. The main contributions of the thesis are (1) generation of the new knowledge regarding the fatigue effect on task performance in the context of home-based rehabilitation, (2) provision of the new fatigue inference system with the highest accuracy in comparison with the existing approaches in literature, and (3) generation of the new knowledge regarding the difference between the individual-based inference and group-based inference approaches

Development and evaluation of a haptic framework supporting telerehabilitation robotics and group interaction

Telerehabilitation robotics has grown remarkably in the past few years. It can provide intensive training to people with special needs remotely while facilitating therapists to observe the whole process. Telerehabilitation robotics is a promising solution supporting routine care which can help to transform face-to-face and one-on-one treatment sessions that require not only intensive human resource but are also restricted to some specialised care centres to treatments that are technology-based (less human involvement) and easy to access remotely from anywhere. However, there are some limitations such as network latency, jitter, and delay of the internet that can affect negatively user experience and quality of the treatment session. Moreover, the lack of social interaction since all treatments are performed over the internet can reduce motivation of the patients. As a result, these limitations are making it very difficult to deliver an efficient recovery plan. This thesis developed and evaluated a new framework designed to facilitate telerehabilitation robotics. The framework integrates multiple cutting-edge technologies to generate playful activities that involve group interaction with binaural audio, visual, and haptic feedback with robot interaction in a variety of environments. The research questions asked were: 1) Can activity mediated by technology motivate and influence the behaviour of users, so that they engage in the activity and sustain a good level of motivation? 2) Will working as a group enhance users’ motivation and interaction? 3) Can we transfer real life activity involving group interaction to virtual domain and deliver it reliably via the internet? There were three goals in this work: first was to compare people’s behaviours and motivations while doing the task in a group and on their own; second was to determine whether group interaction in virtual and reala environments was different from each other in terms of performance, engagement and strategy to complete the task; finally was to test out the effectiveness of the framework based on the benchmarks generated from socially assistive robotics literature. Three studies have been conducted to achieve the first goal, two with healthy participants and one with seven autistic children. The first study observed how people react in a challenging group task while the other two studies compared group and individual interactions. The results obtained from these studies showed that the group interactions were more enjoyable than individual interactions and most likely had more positive effects in terms of user behaviours. This suggests that the group interaction approach has the potential to motivate individuals to make more movements and be more active and could be applied in the future for more serious therapy. Another study has been conducted to measure group interaction’s performance in virtual and real environments and pointed out which aspect influences users’ strategy for dealing with the task. The results from this study helped to form a better understanding to predict a user’s behaviour in a collaborative task. A simulation has been run to compare the results generated from the predictor and the real data. It has shown that, with an appropriate training method, the predictor can perform very well. This thesis has demonstrated the feasibility of group interaction via the internet using robotic technology which could be beneficial for people who require social interaction (e.g. stroke patients and autistic children) in their treatments without regular visits to the clinical centres

Thought-controlled games with brain-computer interfaces

Nowadays, EEG based BCI systems are starting to gain ground in games for health research. With reduced costs and promising an innovative and exciting new interaction paradigm, attracted developers and researchers to use them on video games for serious applications. However, with researchers focusing mostly on the signal processing part, the interaction aspect of the BCIs has been neglected. A gap between classification performance and online control quality for BCI based systems has been created by this research disparity, resulting in suboptimal interactions that lead to user fatigue and loss of motivation over time. Motor-Imagery (MI) based BCIs interaction paradigms can provide an alternative way to overcome motor-related disabilities, and is being deployed in the health environment to promote the functional and structural plasticity of the brain. A BCI system in a neurorehabilitation environment, should not only have a high classification performance, but should also provoke a high level of engagement and sense of control to the user, for it to be advantageous. It should also maximize the level of control on user’s actions, while not requiring them to be subject to long training periods on each specific BCI system. This thesis has two main contributions, the Adaptive Performance Engine, a system we developed that can provide up to 20% improvement to user specific performance, and NeuRow, an immersive Virtual Reality environment for motor neurorehabilitation that consists of a closed neurofeedback interaction loop based on MI and multimodal feedback while using a state-of-the-art Head Mounted Display.Hoje em dia, os sistemas BCI baseados em EEG estão a começar a ganhar terreno em jogos relacionados com a saúde. Com custos reduzidos e prometendo um novo e inovador paradigma de interação, atraiu programadores e investigadores para usá-los em vídeo jogos para aplicações sérias. No entanto, com os investigadores focados principalmente na parte do processamento de sinal, o aspeto de interação dos BCI foi negligenciado. Um fosso entre o desempenho da classificação e a qualidade do controle on-line para sistemas baseados em BCI foi criado por esta disparidade de pesquisa, resultando em interações subótimas que levam à fadiga do usuário e à perda de motivação ao longo do tempo. Os paradigmas de interação BCI baseados em imagética motora (IM) podem fornecer uma maneira alternativa de superar incapacidades motoras, e estão sendo implementados no sector da saúde para promover plasticidade cerebral funcional e estrutural. Um sistema BCI usado num ambiente de neuro-reabilitação, para que seja vantajoso, não só deve ter um alto desempenho de classificação, mas também deve promover um elevado nível de envolvimento e sensação de controlo ao utilizador. Também deve maximizar o nível de controlo nas ações do utilizador, sem exigir que sejam submetidos a longos períodos de treino em cada sistema BCI específico. Esta tese tem duas contribuições principais, o Adaptive Performance Engine, um sistema que desenvolvemos e que pode fornecer até 20% de melhoria para o desempenho específico do usuário, e NeuRow, um ambiente imersivo de Realidade Virtual para neuro-reabilitação motora, que consiste num circuito fechado de interação de neuro-feedback baseado em IM e feedback multimodal e usando um Head Mounted Display de última geração

Mechatronic Systems

Mechatronics, the synergistic blend of mechanics, electronics, and computer science, has evolved over the past twenty five years, leading to a novel stage of engineering design. By integrating the best design practices with the most advanced technologies, mechatronics aims at realizing high-quality products, guaranteeing at the same time a substantial reduction of time and costs of manufacturing. Mechatronic systems are manifold and range from machine components, motion generators, and power producing machines to more complex devices, such as robotic systems and transportation vehicles. With its twenty chapters, which collect contributions from many researchers worldwide, this book provides an excellent survey of recent work in the field of mechatronics with applications in various fields, like robotics, medical and assistive technology, human-machine interaction, unmanned vehicles, manufacturing, and education. We would like to thank all the authors who have invested a great deal of time to write such interesting chapters, which we are sure will be valuable to the readers. Chapters 1 to 6 deal with applications of mechatronics for the development of robotic systems. Medical and assistive technologies and human-machine interaction systems are the topic of chapters 7 to 13.Chapters 14 and 15 concern mechatronic systems for autonomous vehicles. Chapters 16-19 deal with mechatronics in manufacturing contexts. Chapter 20 concludes the book, describing a method for the installation of mechatronics education in schools

A novel parametric scale for determining rehabilitation progress in the upper limb

PhD ThesisThe process of sensorimotor rehabilitation depends upon the clinical condition, age and circumstances of the patient and is unlikely to be continuous or predictable in nature. Mapping progress in conditions such as stroke, cerebral palsy and traumatic brain injury relies upon a variety of qualitative assessments, each resulting in different scales of measurement. Most current assessments are elaborate, specific to certain participants and/or stages of a condition, and subject to inter-rater and intra-rater variability. A simple and reliable measuring system is required that can capture rehabilitation progress from an initial state through to complete rehabilitation. It must be believable, flexible, understandable and accessible if the patient is to benefit from its use. Two-dimensional reaching tasks reflect movements made in typical therapies and activities of daily living. This thesis hypothesises that valuable parameters exist within positional and temporal data gathered from simple reaching tasks. Such parameters should be able to identify movement quality and hence measure state and progress during rehabilitation. They should correlate well with a variety of clinical scales to be meaningful and, as quantifiable measurands, they should be extendable beyond the range of established clinical scales. This thesis proposes a novel solution for the assessment of upper limb rehabilitation. An affordable desktop computer assessment system was developed and used with juvenile patient participants (N=11) to compare simple desktop reaching parameters with a clinical scale. A control group of normal juvenile participants (N=10) provided baseline data. The results indicated good correlation with the clinical scale based upon a weighted combination of pre-selected movement parameters. The methodology developed permits assessment against further clinical scales and additional participant groups allowing rapid, accurate, reliable and extendable assessments. The potential for mass data acquisition from clinical and domestic settings is identified to support the development of further assessments and, potentially, new therapies to address limited therapist availability and innovative treatments

Intelligent sensing technologies for the diagnosis, monitoring and therapy of alzheimer’s disease:A systematic review

Alzheimer’s disease is a lifelong progressive neurological disorder. It is associated with high disease management and caregiver costs. Intelligent sensing systems have the capability to provide context-aware adaptive feedback. These can assist Alzheimer’s patients with, continuous monitoring, functional support and timely therapeutic interventions for whom these are of paramount importance. This review aims to present a summary of such systems reported in the extant literature for the management of Alzheimer’s disease. Four databases were searched, and 253 English language articles were identified published between the years 2015 to 2020. Through a series of filtering mechanisms, 20 articles were found suitable to be included in this review. This study gives an overview of the depth and breadth of the efficacy as well as the limitations of these intelligent systems proposed for Alzheimer’s. Results indicate two broad categories of intelligent technologies, distributed systems and self-contained devices. Distributed systems base their outcomes mostly on long-term monitoring activity patterns of individuals whereas handheld devices give quick assessments through touch, vision and voice. The review concludes by discussing the potential of these intelligent technologies for clinical practice while highlighting future considerations for improvements in the design of these solutions for Alzheimer’s disease

Rehabilitation Engineering

Population ageing has major consequences and implications in all areas of our daily life as well as other important aspects, such as economic growth, savings, investment and consumption, labour markets, pensions, property and care from one generation to another. Additionally, health and related care, family composition and life-style, housing and migration are also affected. Given the rapid increase in the aging of the population and the further increase that is expected in the coming years, an important problem that has to be faced is the corresponding increase in chronic illness, disabilities, and loss of functional independence endemic to the elderly (WHO 2008). For this reason, novel methods of rehabilitation and care management are urgently needed. This book covers many rehabilitation support systems and robots developed for upper limbs, lower limbs as well as visually impaired condition. Other than upper limbs, the lower limb research works are also discussed like motorized foot rest for electric powered wheelchair and standing assistance device

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion