A Kinect-Based Interactive System for Home-Assisted Active Aging

Virtually every country in the world is facing an unprecedented challenge: society is aging.Assistive technologies are expected to play a key role in promoting healthy lifestyles in the elderly.This paper presents a Kinect-based interactive system for home-assisted healthy aging, which guides,supervises, and corrects older users when they perform scheduled physical exercises. Interactionstake place in gamified environments with augmented reality. Many graphical user interface elementsand workflows have been designed considering the sensory, physical and technological shortcomingsof the elderly, adapting accordingly the interaction methods, graphics, exercises, tolerance margins,physical goals, and scoring criteria. Experiments involved 57 participants aged between 65 and 80who performed the same physical routine six times during 15 days. After each session, participantscompleted a usability survey. Results provided significant evidence that support (1) the effectivenessof the system in assisting older users of different age ranges, (2) the accuracy of the system inmeasuring progress in physical achievement of the elderly, and (3) a progressive acceptance ofthe system as it was used. As a main conclusion, the experiments verified that despite their poortechnological skills, older people can adapt positively to the use of an interactive assistance tool foractive aging if they experience clear benefits

Flexible Virtual Reality System for Neurorehabilitation and Quality of Life Improvement

As life expectancy is mostly increasing, the incidence of many neurological disorders is also constantly growing. For improving the physical functions affected by a neurological disorder, rehabilitation procedures are mandatory, and they must be performed regularly. Unfortunately, neurorehabilitation procedures have disadvantages in terms of costs, accessibility and a lack of therapists. This paper presents Immersive Neurorehabilitation Exercises Using Virtual Reality (INREX-VR), our innovative immersive neurorehabilitation system using virtual reality. The system is based on a thorough research methodology and is able to capture real-time user movements and evaluate joint mobility for both upper and lower limbs, record training sessions and save electromyography data. The use of the first-person perspective increases immersion, and the joint range of motion is calculated with the help of both the HTC Vive system and inverse kinematics principles applied on skeleton rigs. Tutorial exercises are demonstrated by a virtual therapist, as they were recorded with real-life physicians, and sessions can be monitored and configured through tele-medicine. Complex movements are practiced in gamified settings, encouraging self-improvement and competition. Finally, we proposed a training plan and preliminary tests which show promising results in terms of accuracy and user feedback. As future developments, we plan to improve the system's accuracy and investigate a wireless alternative based on neural networks.Comment: 47 pages, 20 figures, 17 tables (including annexes), part of the MDPI Sesnsors "Special Issue Smart Sensors and Measurements Methods for Quality of Life and Ambient Assisted Living

Methods and metrics for the improvement of the interaction and the rehabilitation of cerebral palsy through inertial technology

Cerebral palsy (CP) is one of the most limiting disabilities in childhood, with 2.2 cases per 1000 1-year survivors. It is a disorder of movement and posture due to a defect or lesion of the immature brain during the pregnancy or the birth. These motor limitations appear frequently in combination with sensory and cognitive alterations generally result in great difficulties for some people with CP to manipulate objects, communicate and interact with their environment, as well as limiting their mobility. Over the last decades, instruments such as personal computers have become a popular tool to overcome some of the motor limitations and promote neural plasticity, especially during childhood. According to some estimations, 65% of youths with CP that present severely limited manipulation skills cannot use standard mice nor keyboards. Unfortunately, even when people with CP use assistive technology for computer access, they face barriers that lead to the use of typical mice, track balls or touch screens for practical reasons. Nevertheless, with the proper customization, novel developments of alternative input devices such as head mice or eye trackers can be a valuable solution for these individuals. This thesis presents a collection of novel mapping functions and facilitation algorithms that were proposed and designed to ease the act of pointing to graphical elements on the screen—the most elemental task in human-computer interaction—to individuals with CP. These developments were implemented to be used with any head mouse, although they were all tested with the ENLAZA, an inertial interface. The development of such techniques required the following approach: Developing a methodology to evaluate the performance of individuals with CP in pointing tasks, which are usually described as two sequential subtasks: navigation and targeting. Identifying the main motor abnormalities that are present in individuals with CP as well as assessing the compliance of these people with standard motor behaviour models such as Fitts’ law. Designing and validating three novel pointing facilitation techniques to be implemented in a head mouse. They were conceived for users with CP and muscle weakness that have great difficulties to maintain their heads in a stable position. The first two algorithms consist in two novel mapping functions that aim to facilitate the navigation phase, whereas the third technique is based in gravity wells and was specially developed to facilitate the selection of elements in the screen. In parallel with the development of the facilitation techniques for the interaction process, we evaluated the feasibility of use inertial technology for the control of serious videogames as a complement to traditional rehabilitation therapies of posture and balance. The experimental validation here presented confirms that this concept could be implemented in clinical practice with good results. In summary, the works here presented prove the suitability of using inertial technology for the development of an alternative pointing device—and pointing algorithms—based on movements of the head for individuals with CP and severely limited manipulation skills and new rehabilitation therapies for the improvement of posture and balance. All the contributions were validated in collaboration with several centres specialized in CP and similar disorders and users with disability recruited in those centres.La parálisis cerebral (PC) es una de las deficiencias más limitantes de la infancia, con un incidencia de 2.2 casos por cada 1000 supervivientes tras un año de vida. La PC se manifiesta principalmente como una alteración del movimiento y la postura y es consecuencia de un defecto o lesión en el cerebro inmaduro durante el embarazo o el parto. Las limitaciones motrices suelen aparecer además en compañía de alteraciones sensoriales y cognitivas, lo que provoca por lo general grandes dificultades de movilidad, de manipulación, de relación y de interacción con el entorno. En las últimas décadas, el ordenador personal se ha extendido como herramienta para la compensación de parte de estas limitaciones motoras y como medio de promoción de la neuroplasticidad, especialmente durante la infancia. Desafortunadamente, cerca de un 65% de las personas PC que son diagnosticadas con limitaciones severas de manipulación son incapaces de utilizar ratones o teclados convencionales. A veces, ni siquiera la tecnología asistencial les resulta de utilidad ya que se encuentran con impedimentos que hacen que opten por usar dispositivos tradicionales aun sin dominar su manejo. Para estas personas, los desarrollos recientes de ratones operados a través de movimientos residuales con la cabeza o la mirada podrían ser una solución válida, siempre y cuando se personalice su manejo. Esta tesis presenta un conjunto de novedosas funciones de mapeo y algoritmos de facilitaci ón que se han propuesto y diseñado con el ánimo de ayudar a personas con PC en las tareas de apuntamiento de objetos en la pantalla —las más elementales dentro de la interacción con el ordenador. Aunque todas las contribuciones se evaluaron con la interfaz inercial ENLAZA, desarrollada igualmente en nuestro grupo, podrían ser aplicadas a cualquier ratón basado en movimientos de cabeza. El desarrollo de los trabajos se resume en las siguientes tareas abordadas: Desarrollo de una metodología para la evaluación de la habilidad de usuarios con PC en tareas de apuntamiento, que se contemplan como el encadenamiento de dos sub-tareas: navegación (alcance) y selección (clic). Identificación de los tipos de alteraciones motrices presentes en individuos con PC y el grado de ajuste de éstos a modelos estándares de comportamiento motriz como puede ser la ley de Fitts. Propuesta y validación de tres técnicas de facilitación del alcance para ser implementadas en un ratón basado en movimientos de cabeza. La facilitación se ha centrado en personas que presentan debilidad muscular y dificultades para mantener la posición de la cabeza. Mientras que los dos primeros algoritmos se centraron en facilitar la navegación, el tercero tuvo como objetivo ayudar en la selección a través de una técnica basada en pozos gravitatorios de proximidad. En paralelo al desarrollo de estos algoritmos de facilitación de la interacción, evaluamos la posibilidad de utilizar tecnología inercial para el control de videojuegos en rehabilitación. Nuestra validación experimental demostró que este concepto puede implementarse en la práctica clínica como complemento a terapias tradicionales de rehabilitación de la postura y el equilibrio. Como conclusión, los trabajos desarrollados en esta tesis vienen a constatar la idoneidad de utilizar sensores inerciales para el desarrollo de interfaces de accesso alternativo al ordenador basados en movimientos residuales de la cabeza para personas con limitaciones severas de manipulación. Esta solución se complementa con algoritmos de facilitación del alcance. Por otra parte, estas soluciones tecnológicas de interfaz con el ordenador representan igualmente un complemento de terapias tradicionales de rehabilitación de la postura y el equilibrio. Todas las contribuciones se validaron en colaboración con una serie de centros especializados en parálisis cerebral y trastornos afines contando con usuarios con discapacidad reclutados en dichos centros.This thesis was completed in the Group of Neural and Cognitive Engineering (gNEC) of the CAR UPM-CSIC with the financial support of the FP7 Framework EU Research Project ABC (EU-2012-287774), the IVANPACE Project (funded by Obra Social de Caja Cantabria, 2012-2013), and the Spanish Ministry of Economy and Competitiveness in the framework of two projects: the Interplay Project (RTC-2014-1812-1) and most recently the InterAAC Project (RTC-2015-4327-1)Programa Oficial de Doctorado en Ingeniería Eléctrica, Electrónica y AutomáticaPresidente: Juan Manuel Belda Lois.- Secretario: María Dolores Blanco Rojas.- Vocal: Luis Fernando Sánchez Sante

Novel Bidirectional Body - Machine Interface to Control Upper Limb Prosthesis

Objective. The journey of a bionic prosthetic user is characterized by the opportunities and limitations involved in adopting a device (the prosthesis) that should enable activities of daily living (ADL). Within this context, experiencing a bionic hand as a functional (and, possibly, embodied) limb constitutes the premise for mitigating the risk of its abandonment through the continuous use of the device. To achieve such a result, different aspects must be considered for making the artificial limb an effective support for carrying out ADLs. Among them, intuitive and robust control is fundamental to improving amputees’ quality of life using upper limb prostheses. Still, as artificial proprioception is essential to perceive the prosthesis movement without constant visual attention, a good control framework may not be enough to restore practical functionality to the limb. To overcome this, bidirectional communication between the user and the prosthesis has been recently introduced and is a requirement of utmost importance in developing prosthetic hands. Indeed, closing the control loop between the user and a prosthesis by providing artificial sensory feedback is a fundamental step towards the complete restoration of the lost sensory-motor functions. Within my PhD work, I proposed the development of a more controllable and sensitive human-like hand prosthesis, i.e., the Hannes prosthetic hand, to improve its usability and effectiveness. Approach. To achieve the objectives of this thesis work, I developed a modular and scalable software and firmware architecture to control the Hannes prosthetic multi-Degree of Freedom (DoF) system and to fit all users’ needs (hand aperture, wrist rotation, and wrist flexion in different combinations). On top of this, I developed several Pattern Recognition (PR) algorithms to translate electromyographic (EMG) activity into complex movements. However, stability and repeatability were still unmet requirements in multi-DoF upper limb systems; hence, I started by investigating different strategies to produce a more robust control. To do this, EMG signals were collected from trans-radial amputees using an array of up to six sensors placed over the skin. Secondly, I developed a vibrotactile system to implement haptic feedback to restore proprioception and create a bidirectional connection between the user and the prosthesis. Similarly, I implemented an object stiffness detection to restore tactile sensation able to connect the user with the external word. This closed-loop control between EMG and vibration feedback is essential to implementing a Bidirectional Body - Machine Interface to impact amputees’ daily life strongly. For each of these three activities: (i) implementation of robust pattern recognition control algorithms, (ii) restoration of proprioception, and (iii) restoration of the feeling of the grasped object's stiffness, I performed a study where data from healthy subjects and amputees was collected, in order to demonstrate the efficacy and usability of my implementations. In each study, I evaluated both the algorithms and the subjects’ ability to use the prosthesis by means of the F1Score parameter (offline) and the Target Achievement Control test-TAC (online). With this test, I analyzed the error rate, path efficiency, and time efficiency in completing different tasks. Main results. Among the several tested methods for Pattern Recognition, the Non-Linear Logistic Regression (NLR) resulted to be the best algorithm in terms of F1Score (99%, robustness), whereas the minimum number of electrodes needed for its functioning was determined to be 4 in the conducted offline analyses. Further, I demonstrated that its low computational burden allowed its implementation and integration on a microcontroller running at a sampling frequency of 300Hz (efficiency). Finally, the online implementation allowed the subject to simultaneously control the Hannes prosthesis DoFs, in a bioinspired and human-like way. In addition, I performed further tests with the same NLR-based control by endowing it with closed-loop proprioceptive feedback. In this scenario, the results achieved during the TAC test obtained an error rate of 15% and a path efficiency of 60% in experiments where no sources of information were available (no visual and no audio feedback). Such results demonstrated an improvement in the controllability of the system with an impact on user experience. Significance. The obtained results confirmed the hypothesis of improving robustness and efficiency of a prosthetic control thanks to of the implemented closed-loop approach. The bidirectional communication between the user and the prosthesis is capable to restore the loss of sensory functionality, with promising implications on direct translation in the clinical practice

ReLiPh: rehabilitation for lower limb with phantom pain

O membro fantasma ou sensação fantasma, ao longo dos anos, têm se destacado por ser originada de diferentes causas. Pesquisas e estudos mostram que, após uma amputação, na maioria dos casos, experienciam a sensação de membro fantasma e em muitos desses casos dolorosos. A presente tese baseia-se em uma pequena parte deste amplo tópico. Baseada na terapia de espelho usada na reabilitação e tratamento da dor fantasma. Ao longo do projeto, nós estudamos quais são os componentes mais relevantes para a reorganização/reestruturação, a fim de eliminar os sintomas negativos e futuros distúrbios/perturbações. Associada na relação do cérebro com o corpo, ou seja, as imagens formadas pelo cérebro em relação ao corpo físico desempenhando um papel crucial na relação do membro fantasma e da dor no membro fantasma, bem como no processo de cura e tratamento através de exercícios e no relacionamento da imagem que o cérebro tem do corpo físico. Esta dissertação tem como objetivo na construção de uma nova abordagem tecnológica, baseando-se nos princípios e critérios utilizados na terapia de espelho. A metodologia assenta na criação de um ambiente de realidade virtual controlado por um dispositivo que captura a atividade muscular em tempo real. Implementado num jogo baseado em movimentos/exercícios simples e naturais, sem uso de força ou esforço. Os elevados resultados verificados e testados, em indivíduos saudáveis e em um estudo de caso, na redução da dor fantasma, gerando um interesse e motivação, além de um melhor senso de presença e foco durante o seu uso. Concluindo, o projeto abre novas direções futuras de como novas abordagens tecnológicas podem ser usados nas pesquisas médicas na área do membro e na dor fantasma, em ambientes controlados e contextualizados. Melhorando a eficácia e eficiência, garantindo uma maior flexibilidade nos diferentes casos de amputação.The phantom limb or phantom sensation, over the years, has stood out being originated from different causes. Research and studies show that after an amputation, in most cases, they experience the sensation of a phantom limb and in many of those painful feelings. This thesis is based on a small part of this wide topic. Based on the mirror therapy used in rehabilitation and treatment for phantom pain. Throughout the project, we study what are the most relevant components to reorganization/restructuring in order to eliminate negative symptoms and future disturbances. Moreover it is established in the relationship of the brain with the body, that the images formed by the brain in relation to the physical body play a crucial role in the relationship with the phantom limb and phantom limb pain, as well as in the process of healing and treatment throughout exercises and the relationship of the image that the brain has to the physical body. This dissertation aims to build a new technological approach, based on the principles and criteria used in mirror therapy. The methodology is based on the creation of a virtual reality environment controlled by a device which captures the muscle activity in real time. Implemented in a game based on natural and simple effortless exercises without the use of strength. The high results verified and tested, in healthy subjects and in a case study, to reduce phantom pain, generating an interest and motivation, as well as a better sense of presence and focus during its use. In conclusion, the project opens up new future directions of how new technological approaches can be used in medical research in the field of phantom limbs and in phantom pain, in a controlled and contextualized environments and/or movements. Improving effectiveness and efficiency ensuring greater flexibility in different cases of amputation

Rehabilitation process using electromyography and biofeedback

Treballs Finals de Grau d'Enginyeria Biomèdica. Facultat de Medicina i Ciències de la Salut. Universitat de Barcelona. Curs: 2020-2021. Director: José Luis Parreño Catalan. Tutor: Manel Puig Vidal.A good rehabilitation routine is essential for the best possible recovery after an injury or to increase the quality of life of those who suffer from neuromusculoskeletal diseases. It is of particular relevance to maintain the motivation throughout all the process, for which videogames may play an essential role. Biofeedback is a process that provides real time information from psychophysiological recordings about the levels at which physiological systems are functioning. In this project the biofeedback system has been implemented by means of a low-cost EMG system created using Arduino. The EMG system has been developed using an Olimexino-328 microcontroller and an EMG-Shield, both from Olimex. The program was developed using Arduino IDE. To assess the quality of the signal of the prototype, it was compared to a professional EMG device, the DataLog from Biometrics Ltd. The comparison showed promising results although it could be improved by means of post-processing algorithms. An audio-visual Biofeedback system targeting maximum strength and explosiveness of the muscles was created using Python. Flappy Bird game commands were changed to control them with the EMG low-cost prototype. The flap logic of the game is guided by a threshold fixed automatically by the software at the 60% of the maximum signal obtained by the EMG system. The system was tested to optimize its performance and fix possible flaws. Although it is simple and further investigation may be needed, having in mind the fewer resources used, the system performance is encouraging, and a clinical trial should be performed to assess its real behaviour, usefulness, and efficiency for rehabilitation purposes

A scoping review and a taxonomy of the use of motion-based technology centered on the end user. A special focus on elderly health

Motion-based technology (MBT) has been applied in the last decades with enormous success in a high number of applications. Its use continues growing and is specially interesting in the health area. Nowadays, its employment is being more and more specialised with respect to the profile of the end user (i.e., child, adolescent/teenager, adult or elderly). This paper first reviews the use of MBT centered in the end user from a global perspective. It also proposes a taxonomy that allows cataloguing the MBT employment directed to the end user. Then, from these results, the paper centers the review on the MBT application aiming to improve the health of elderly. The results highlighted in this paper can help to a better understanding of MBT, especially when it is applied thinking in elderly as the end users.This study is partially funded by the Universidad de Málaga with the national project Bio4Res (PID2021-125184NB-I00) from the Ministerio de Ciencia e Innovaci ́on de Espa ̃na (MCIN). Funding for open access charge: Universidad de Málaga / CBUA