Future Trends of Virtual, Augmented Reality, and Games for Health

Serious game is now a multi-billion dollar industry and is still growing steadily in many sectors. As a major subset of serious games, designing and developing Virtual Reality (VR), Augmented Reality (AR), and serious games or adopting off-the-shelf games to support medical education, rehabilitation, or promote health has become a promising frontier in the healthcare sector since 2004, because games technology is inexpensive, widely available, fun and entertaining for people of all ages, with various health conditions and different sensory, motor, and cognitive capabilities. In this chapter, we provide the reader an overview of the book with a perspective of future trends of VR, AR simulation and serious games for healthcare

Smart sensing and AI for physical therapy in IoT era

It is well known that medical spending increase with disability status. Per capita spending for people with five or more limitations in activities of daily living (ADLs) is nearly five times the amount incurred by those with limitations in only one instrumental activities of daily living (IADLs). Physical therapy is the way to improve the motor capabilities however it takes a lot of time, it requires physiotherapists services, is often painful and the outcome are evaluated in subjective way. New technologies including smart sensors were adopted in healthcare including wearable solutions for cardiac and respiratory activity monitoring and successfully are contributing to reduce the costs of services. In the case of motor activity and particularly in physical rehabilitation the developments are still reduced the physical therapy services are using as hardware mechanical equipment without sensing, embedded processing and internet connectivity that significatively reduce the possibility to measure and evaluate the physical training outcomes in objective way. In this paper the disruptive solutions for physical therapy are presented that are based on hot technologies such as smart sensors, IoT, virtual reality (VR), mixed reality (MR), and artificial intelligence (AI). Applied AI may conduct to develop models, classifiers (gait classification) and short term or medium term prediction of physical therapy outcomes. Highly motivation of the patients under physical rehabilitation can be increased promoting serious game characterized by VR and MR scenariosinfo:eu-repo/semantics/publishedVersio

Wearable and IoT technologies application for physical rehabilitation

This research consists in the development an IoT Physical Rehabilitation solution based on wearable devices, combining a set of smart gloves and smart headband for use in natural interactions with a set of VR therapeutic serious games developed on the Unity 3D gaming platform. The system permits to perform training sessions for hands and fingers motor rehabilitation. Data acquisition is performed by Arduino Nano Microcontroller computation platform with ADC connected to the analog measurement channels materialized by piezo-resistive force sensors and connected to an IMU module via I2C. Data communication is performed using the Bluetooth wireless communication protocol. The smart headband, designed to be used as a first- person-controller in game scenes, will be responsible for collecting the patient's head rotation value, this parameter will be used as the player's avatar head rotation value, approaching the user and the virtual environment in a semi-immersive way. The acquired data are stored and processed on a remote server, which will help the physiotherapist to evaluate the patients' performance around the different physical activities during a rehabilitation session, using a Mobile Application developed for the configuration of games and visualization of results. The use of serious games allows a patient with motor impairments to perform exercises in a highly interactive and non-intrusive way, based on different scenarios of Virtual Reality, contributing to increase the motivation during the rehabilitation process. The system allows to perform an unlimited number of training sessions, making possible to visualize historical values and compare the results of the different performed sessions, for objective evolution of rehabilitation outcome. Some metrics associated with upper limb exercises were also considered to characterize the patient’s movement during the session.Este trabalho de pesquisa consiste no desenvolvimento de uma solução de Reabilitação Física IoT baseada em dispositivos de vestuário, combinando um conjunto de luvas inteligentes e uma fita-de-cabeça inteligente para utilização em interações naturais com um conjunto de jogos terapêuticos sérios de Realidade Virtual desenvolvidos na plataforma de jogos Unity 3D. O sistema permite realizar sessões de treino para reabilitação motora de mãos e dedos. A aquisição de dados é realizada pela plataforma de computação Arduino utilizando um Microcontrolador Nano com ADC (Conversor Analógico-Digital) conectado aos canais de medição analógicos materializados por sensores de força piezo-resistivos e a um módulo IMU por I2C. A comunicação de dados é realizada usando o protocolo de comunicação sem fio Bluetooth. A fita-de-cabeça inteligente, projetada para ser usada como controlador de primeira pessoa nos cenários de jogo, será responsável por coletar o valor de rotação da cabeça do paciente, esse parâmetro será usado como valor de rotação da cabeça do avatar do jogador, aproximando o utilizador e o ambiente virtual de forma semi-imersiva. Os dados adquiridos são armazenados e processados num servidor remoto, o que ajudará o fisioterapeuta a avaliar o desempenho dos pacientes em diferentes atividades físicas durante uma sessão de reabilitação, utilizando uma Aplicação Móvel desenvolvido para configuração de jogos e visualização de resultados. A utilização de jogos sérios permite que um paciente com deficiências motoras realize exercícios de forma altamente interativa e não intrusiva, com base em diferentes cenários de Realidade Virtual, contribuindo para aumentar a motivação durante o processo de reabilitação. O sistema permite realizar um número ilimitado de sessões de treinamento, possibilitando visualizar valores históricos e comparar os resultados das diferentes sessões realizadas, para a evolução objetiva do resultado da reabilitação. Algumas métricas associadas aos exercícios dos membros superiores também foram consideradas para caracterizar o movimento do paciente durante a sessão

PhysioAR: smart sensing and augmented reality for physical rehabilitation

The continuous evolution of technology allows for a better analysis of the human being. In certain medical areas such as physiotherapy is required a correct analysis of the patient's evolution. The development of Information and Communication Technologies and recent innovations in the Internet of Things opens new possibilities in the medical field as systems of remote monitoring of patients with new sensors that allow the correct analysis of the health data of patients. In physiotherapy one of the most common problems in the application of treatments is the patient demotivation, something that today can be reduced with the introduction of Augmented Reality that provides a new experience to the patient. For this purpose, a system was developed that combines intelligent sensors with Augmented Reality application that will help monitor patient performance. This system is capable of monitoring lower limb movements acceleration, knee joint angle, patient equilibrium, muscular activity and cardiac activity using electromyography and electrocardiography with a wearable set of tools for easy utilization.A evolução continua da tecnologia permite cada vez mais uma melhor análise do ser humano. Em certas áreas médicas, como a fisioterapia, é necessária uma correta análise da evolução do paciente. O desenvolvimento das Tecnologias de Informação e Comunicação, e as inovações no domínio de Internet das Coisas novas possibilidades no ramo da medicina, como sistemas de monitorização remota de pacientes com novos sensores que permitem a correta análise dos dados de saúde dos pacientes. Na fisioterapia um dos problemas mais comuns na aplicação dos tratamentos é a desmotivação do paciente, algo que hoje pode ser reduzido com introdução da aplicação da Realidade Aumentada que proporciona uma nova experiência ao paciente. Para isso nesta dissertação foi desenvolvido um sistema que combina sensores inteligentes com Realidade Aumentada que vai ajudar o paciente monitorizando o seu desempenho. Este sistema é capaz de monitorizar o ângulo do joelho, captar acelaração de movimentos dos membros inferiores, equilíbrio do paciente, atividade muscular e atividade cárdica usando electromiografia e electrocardiografia num conjunto wearable de fácil utilização

Serious game augmented reality 3D for physical rehabilitation

This research consists in the development of a PhysioAR framework (Augmented Reality Physiotherapy) that consider a set of two wearable sensors (Left Controller and Right Controller and Meta/Oculus Quest headset controller for use in natural interactions with a set of AR therapeutic serious games developed on the Unity 3D. The system allows to perform training sessions for hands and fingers, knees and legs motor rehabilitation bearing in mind that the games are for people who have suffered from stroke. The training is part of special care that must be taken for this through the serious games that are properly adapted to be a source of motivation and easy to be played. This FisioAR project includes, two different apps designed, one for calendar and for physiotherapists has a background data with all information needed to do and other to make login in main app and have the possibility to interact with our three types of games specifically designed, developed and implemented for Oculus Quest. Two different mobile apps were constructed on Outsystems platform, where one is destinated to physiotherapists and other is destinated to AVC patient’s. Three Different types of serious games were developed on Unity Platform Engine and all the games have specific contents to be played according with motor and cognitive rehabilitation objectives. The first game called Boxes Game, has six cubes displayed with different colors and six spheres also with six different colors. The main goal of this game is to put the maximum number of spheres in a box with the same color. This game will involve the use of legs, knees and arms and can be easily adapted to each patients’ conditions, making it more or less demanding. The Second Game is called Garden Care Game. Its scenario was made with prefabs (assets) and materials from Unity asset store to simulate a realistic garden, with a watering can, fences and a set of flowers. The main goal of this game is to care the flowers with water. This simple goal is related with the measurement of the wrist rotation made by the patient through wearable sensors while watering each flower. This game as a score for each flower watered. In the Third Game called Puzzle Game, there’s a white screen with the same number of divisions as the existing image blocks in project.Esta pesquisa consiste no desenvolvimento de uma solução do projeto FisioAR baseada em dispositivos vestíveis, combinando um conjunto de sensores vestíveis e controlador de headset para uso em interações naturais com um conjunto de serious games terapêuticos VR desenvolvidos na plataforma de games 3D Unity. O sistema permite realizar treinos de reabilitação motora de mãos e dedos, joelhos e pernas tendo em vista que os jogos são para pessoas que sofreram AVC e devem ser tomados cuidados especiais com isso e que os jogos estão devidamente adaptados para serem mais simples. ser jogado. Este projeto FisioAR tem em todas as implementações, dois aplicativos diferentes projetados, três tipos diferentes de jogos projetados no Oculus Quest. Dois aplicativos diferentes foram construídos na plataforma Outsystems sendo um destinado a fisioterapeutas e outro a pacientes AVC. Três tipos diferentes de jogos foram especialmente projetados no Unity Platform Engine e todos os jogos possuem conteúdos específicos para serem jogados. O primeiro jogo denominado Boxes Game, tem seis cubos apresentados com cores diferentes e seis esferas também com seis cores diferentes. O principal objetivo deste jogo é colocar o número máximo de esferas em uma caixa com a mesma cor e com distância mínima percorrida. Este jogo envolverá o uso de pernas, joelhos e braços e pode ser facilmente adaptado às condições de cada paciente, tornando-o mais ou menos exigente. O segundo jogo é chamado de jogo de cuidado de jardim. Seu cenário foi feito com pré-fabricados e materiais da loja de ativos da unidade para simular um jardim realista, com regador, cercas e um conjunto de flores. O objetivo principal deste jogo é regar as flores. Esse objetivo simples está relacionado à medição da rotação do punho feita pelo paciente por meio de sensores vestíveis ao regar cada flor. Este jogo é uma pontuação para cada flor regada. No terceiro jogo, chamado Puzzle Game, há uma tela branca com o mesmo número de divisões que os blocos de imagem existentes no projeto

Fall prevention intervention technologies: A conceptual framework and survey of the state of the art

In recent years, an ever increasing range of technology-based applications have been developed with the goal of assisting in the delivery of more effective and efficient fall prevention interventions. Whilst there have been a number of studies that have surveyed technologies for a particular sub-domain of fall prevention, there is no existing research which surveys the full spectrum of falls prevention interventions and characterises the range of technologies that have augmented this landscape. This study presents a conceptual framework and survey of the state of the art of technology-based fall prevention systems which is derived from a systematic template analysis of studies presented in contemporary research literature. The framework proposes four broad categories of fall prevention intervention system: Pre-fall prevention; Post-fall prevention; Fall injury prevention; Cross-fall prevention. Other categories include, Application type, Technology deployment platform, Information sources, Deployment environment, User interface type, and Collaborative function. After presenting the conceptual framework, a detailed survey of the state of the art is presented as a function of the proposed framework. A number of research challenges emerge as a result of surveying the research literature, which include a need for: new systems that focus on overcoming extrinsic falls risk factors; systems that support the environmental risk assessment process; systems that enable patients and practitioners to develop more collaborative relationships and engage in shared decision making during falls risk assessment and prevention activities. In response to these challenges, recommendations and future research directions are proposed to overcome each respective challenge.The Royal Society, grant Ref: RG13082

Augmented reality system with application in physical rehabilitation

The aging phenomenon causes increased physiotherapy services requirements, with increased costs associated with long rehabilitation periods. Traditional rehabilitation methods rely on the subjective assessment of physiotherapists without supported training data. To overcome the shortcoming of traditional rehabilitation method and improve the efficiency of rehabilitation, AR (Augmented Reality) which represents a promissory technology that provides an immersive interaction with real and virtual objects is used. The AR devices may assure the capture body posture and scan the real environment that conducted to a growing number of AR applications focused on physical rehabilitation. In this MSc thesis, an AR platform used to materialize a physical rehabilitation plan for stroke patients is presented. Gait training is a significant part of physical rehabilitation for stroke patients. AR represents a promissory solution for training assessment providing information to the patients and physiotherapists about exercises to be done and the reached results. As part of MSc work an iOS application was developed in unity 3D platform. This application immersing patients in a mixed environment that combine real-world and virtual objects. The human computer interface is materialized by an iPhone as head-mounted 3D display and a set of wireless sensors for physiological and motion parameters measurement. The position and velocity of the patient are recorded by a smart carpet that includes capacitive sensors connected to a computation unit characterized by Wi-Fi communication capabilities. AR training scenario and the corresponding experimental results are part of the thesis.O envelhecimento causa um aumento das necessidades dos serviços de fisioterapia, com aumento dos custos associados a longos períodos de reabilitação. Os métodos tradicionais de reabilitação dependem da avaliação subjetiva de fisioterapeutas sem registo automatizado de dados de treino. Com o principal objetivo de superar os problemas do método tradicional e melhorar a eficiência da reabilitação, é utilizada a RA (Realidade Aumentada), que representa uma tecnologia promissora, que fornece uma interação imersiva com objetos reais e virtuais. Os dispositivos de RA são capazes de garantir uma postura correta do corpo de capturar e verificar o ambiente real, o que levou a um número crescente de aplicações de RA focados na reabilitação física. Neste projeto, é apresentada uma plataforma de RA, utilizada para materializar um plano de reabilitação física para pacientes que sofreram AVC. O treino de marcha é uma parte significativa da reabilitação física para pacientes com AVC. A RA apresenta-se como uma solução promissora para a avaliação do treino, fornecendo informações aos pacientes e aos profissionais de fisioterapia sobre os exercícios a serem realizados e os resultados alcançados. Como parte deste projeto, uma aplicação iOS foi desenvolvida na plataforma Unity 3D. Esta aplicação fornece aos pacientes um ambiente imersivo que combina objetos reais e virtuais. A interface de RA é materializada por um iPhone montado num suporte de cabeça do utilizador, assim como um conjunto de sensores sem fios para medição de parâmetros fisiológicos e de movimento. A posição e a velocidade do paciente são registadas por um tapete inteligente que inclui sensores capacitivos conectados a uma unidade de computação, caracterizada por comunicação via Wi-Fi. O cenário de treino em RA e os resultados experimentais correspondentes fazem parte desta dissertação

Fall Prediction and Prevention Systems: Recent Trends, Challenges, and Future Research Directions.

Fall prediction is a multifaceted problem that involves complex interactions between physiological, behavioral, and environmental factors. Existing fall detection and prediction systems mainly focus on physiological factors such as gait, vision, and cognition, and do not address the multifactorial nature of falls. In addition, these systems lack efficient user interfaces and feedback for preventing future falls. Recent advances in internet of things (IoT) and mobile technologies offer ample opportunities for integrating contextual information about patient behavior and environment along with physiological health data for predicting falls. This article reviews the state-of-the-art in fall detection and prediction systems. It also describes the challenges, limitations, and future directions in the design and implementation of effective fall prediction and prevention systems

Smart sensing systems for in-home health status and emotional well-being monitoring during COVID-19

The COVID-19 pandemic has restricted the mobility of the population. The experts propose several solutions in order to decrease the number of patients infected with this new virus by treating and monitoring them within the comfort of their own home. A new direction for the research has been identified including healthcare smart sensing systems which can provide medical diagnoses, surveillance, and treatment partially or totally remotely. The field of wearable, smart sensing solutions is becoming nowadays a widely accepted solution characterized also by the increased level of acceptance with regard to home health status monitoring. Pervasive computing and wearable solutions are frequently a topic included in current projects and are expected in new future developments, particularly in the pandemic context which forces people to remain mostly at home. As part of wearable devices the design of textiles, computer science, and smart materials are the three major development directions. The latest developments associated with the monitoring of health status and emotional well-being are presented and discussed in this chapter.info:eu-repo/semantics/submittedVersio