Recognition of elementary upper limb movements in an activity of daily living using data from wrist mounted accelerometers

In this paper we present a methodology as a proof of concept for recognizing fundamental movements of the humanarm (extension, flexion and rotation of the forearm) involved in ‘making-a-cup-of-tea’, typical of an activity of daily-living (ADL). The movements are initially performed in a controlled environment as part of a training phase and the data are grouped into three clusters using k-means clustering. Movements performed during ADL, forming part of the testing phase, are associated with each cluster label using a minimum distance classifier in a multi-dimensional feature space, comprising of features selected from a ranked set of 30 features, using Euclidean and Mahalonobis distance as the metric. Experiments were performed with four healthy subjects and our results show that the proposed methodology can detect the three movements with an overall average accuracy of 88% across all subjects and arm movement types using Euclidean distance classifier

Heterogeneous multi-modal sensor fusion with hybrid attention for exercise recognition.

Exercise adherence is a key component of digital behaviour change interventions for the self-management of musculoskeletal pain. Automated monitoring of exercise adherence requires sensors that can capture patients performing exercises and Machine Learning (ML) algorithms that can recognise exercises. In contrast to ambulatory activities that are recognisable with a wrist accelerometer data; exercises require multiple sensor modalities because of the complexity of movements and the settings involved. Exercise Recognition (ExR) pose many challenges to ML researchers due to the heterogeneity of the sensor modalities (e.g. image/video streams, wearables, pressure mats). We recently published MEx, a benchmark dataset for ExR, to promote the study of new and transferable HAR methods to improve ExR and benchmarked the state-of-the-art ML algorithms on 4 modalities. The results highlighted the need for fusion methods that unite the individual strengths of modalities. In this paper we explore fusion methods with focus on attention and propose a novel multi-modal hybrid attention fusion architecture mHAF for ExR. We achieve the best performance of 96.24% (F1-measure) with a modality combination of a pressure mat, a depth camera and an accelerometer on the thigh. mHAF significantly outperforms multiple baselines and the contribution of model components are verified with an ablation study. The benefits of attention fusion are clearly demonstrated by visualising attention weights; showing how mHAF learns feature importance and modality combinations suited for different exercise classes. We highlight the importance of improving deployability and minimising obtrusiveness by exploring the best performing 2 and 3 modality combinations

Wearables for independent living in older adults: Gait and falls

Solutions are needed to satisfy care demands of older adults to live independently. Wearable technology (wearables) is one approach that offers a viable means for ubiquitous, sustainable and scalable monitoring of the health of older adults in habitual free-living environments. Gait has been presented as a relevant (bio)marker in ageing and pathological studies, with objective assessment achievable by inertial-based wearables. Commercial wearables have struggled to provide accurate analytics and have been limited by non-clinically oriented gait outcomes. Moreover, some research-grade wearables also fail to provide transparent functionality due to limitations in proprietary software. Innovation within this field is often sporadic, with large heterogeneity of wearable types and algorithms for gait outcomes leading to a lack of pragmatic use. This review provides a summary of the recent literature on gait assessment through the use of wearables, focusing on the need for an algorithm fusion approach to measurement, culminating in the ability to better detect and classify falls. A brief presentation of wearables in one pathological group is presented, identifying appropriate work for researchers in other cohorts to utilise. Suggestions for how this domain needs to progress are also summarised

Artificial Intelligence for skeleton-based physical rehabilitation action evaluation: A systematic review

Performing prescribed physical exercises during home-based rehabilitation programs plays an important role in regaining muscle strength and improving balance for people with different physical disabilities. However, patients attending these programs are not able to assess their action performance in the absence of a medical expert. Recently, vision-based sensors have been deployed in the activity monitoring domain. They are capable of capturing accurate skeleton data. Furthermore, there have been significant advancements in Computer Vision (CV) and Deep Learning (DL) methodologies. These factors have promoted the solutions for designing automatic patient’s activity monitoring models. Then, improving such systems’ performance to assist patients and physiotherapists has attracted wide interest of the research community. This paper provides a comprehensive and up-to-date literature review on different stages of skeleton data acquisition processes for the aim of physio exercise monitoring. Then, the previously reported Artificial Intelligence (AI) - based methodologies for skeleton data analysis will be reviewed. In particular, feature learning from skeleton data, evaluation, and feedback generation for the purpose of rehabilitation monitoring will be studied. Furthermore, the associated challenges to these processes will be reviewed. Finally, the paper puts forward several suggestions for future research directions in this area

The KIMORE dataset: KInematic assessment of MOvement and clinical scores for remote monitoring of physical REhabilitation

The paper proposes a free dataset, available at the following link1, named KIMORE, regarding different rehabilitation exercises collected by a RGB-D sensor. Three data inputs including RGB, Depth videos and skeleton joint positions were recorded during five physical exercises, specific for low back pain and accurately selected by physicians. For each exercise, the dataset also provides a set of features, specifically defined by the physicians, and relevant to describe its scope. These features, validated with respect to a stereophotogrammetric system, can be analyzed to compute a score for the subject's performance. The dataset also contains an evaluation of the same performance provided by the clinicians, through a clinical questionnaire. The impact of KIMORE has been analyzed by comparing the output obtained by an example of rule and template-based approaches and the clinical score. The dataset presented is intended to be used as a benchmark for human movement assessment in a rehabilitation scenario in order to test the effectiveness and the reliability of different computational approaches. Unlike other existing datasets, the KIMORE merges a large heterogeneous population of 78 subjects, divided into 2 groups with 44 healthy subjects and 34 with motor dysfunctions. It provides the most clinically-relevant features and the clinical score for each exercise

A database of physical therapy exercises with variability of execution collected by wearable sensors

This document introduces the PHYTMO database, which contains data from physical therapies recorded with inertial sensors, including information from an optical reference system. PHYTMO includes the recording of 30 volunteers, aged between 20 and 70 years old. A total amount of 6 exercises and 3 gait variations were recorded. The volunteers performed two series with a minimum of 8 repetitions in each one. PHYTMO includes magneto-inertial data, together with a highly accurate location and orientation in the 3D space provided by the optical system. The files were stored in CSV format to ensure its usability. The aim of this dataset is the availability of data for two main purposes: the analysis of techniques for the identification and evaluation of exercises using inertial sensors and the validation of inertial sensor-based algorithms for human motion monitoring. Furthermore, the database stores enough data to apply Machine Learning-based algorithms. The participants' age range is large enough to establish age-based metrics for the exercises evaluation or the study of differences in motions between different groups

Augmented Human Assistance (AHA)

Aging and sedentarism are two main challenges for social and health systems in modern societies. To face these challenges a new generation of ICT based solutions is being developed to promote active aging, prevent sedentarism and find new tools to support the large populations of patients that suffer chronic conditions as result of aging. Such solutions have the potential to transform healthcare by optimizing resource allocation, reducing costs, improving diagno ses and enabling novel therapies, thus increasing quality of life. The primary goal of the “AHA: Augmented Human Assistance” project is to de velop novel assistive technologies to promote exercise among the elderly and patients of motor disabilities. For exercise programs to be effective, it is essential that users and patients comply with the prescribed schedule and perform the ex ercises following established protocols. Until now this has been achieved by hu man monitoring in rehabilitation and therapy session, where the clinicians or therapists permanently accompany users or patient. In many cases, exercises are prescribed for home performance, in which case it is not possible to validate their execution. In this context, the AHA project is an integrative and cross-discipli nary approach of 4 Portuguese universities, the CMU, and 2 Portuguese industry partners, that combines innovation and fundamental research in the areas of hu man-computer interaction, robotics, serious games and physiological computing (see partner list in Appendix A). In the project, we capitalize on recent innova tions and aim at enriching the capabilities and range of application of assistive devices via the combination of (1) assistive robotics; (2) technologies that use well-understood motivational techniques to induce people to do their exercises in the first place, and to do them correctly and completely; (3) tailored and relevant guidance in regard to health care and social support and activities; and (4) tech nologies to self-monitoring and sharing of progress with health-care provider enabling clinicians to fine-tune the exercise regimen to suit the participant’s ac tual progress. We highlight the development of a set of exergames (serious games controlled by the movement of the user’s body limbs) specifically designed for the needs of the target population according to best practices in sports and human kinetics sciences. The games can be adapted to the limitations of the users (e.g. to play in a sitting position) so a large fraction of the population can benefit from them. The games can be executed with biofeedback provided from wearable sensors, to pro duce more controlled exercise benefits. The games can be played in multi-user settings, either in cooperative or competitive mode, to promote the social rela tions among players. The games contain regional motives to trigger memories from the past and other gamification techniques that keep the users involved in the exercise program. The games are projected in the environment through aug mented reality techniques that create a more immersive and engaging experience than conventional displays. Virtual coach techniques are able to monitor the cor rectness of the exercise and provide immediate guidance to the user, as well as providing reports for therapists. A socially assistive robot can play the role of the coach and provide an additional socio-cognitive dimension to the experience to complement the role of the therapist. A web service that records the users’ per formances and allows the authorized therapists to access and configure the exer cise program provides a valuable management tool for caregivers and clinical staff. It can also provide a social network for players, increasing adherence to the therapies. We have performed several end-user studies that validate the proposed ap proaches. Together, or in isolation, these solutions provide users, caregivers, health professionals and institutions, valuable tools for health promotion, disease monitoring and prevention.info:eu-repo/semantics/publishedVersio

Kinect-based Solution for the Home Monitoring of Gait and Balance in Elderly People with and without Neurological Diseases

Alterations of gait and balance are a significant cause of falls, injuries, and consequent hospitalizations in the elderly. In addition to age-associated motor decline, other factors can impact gait and stability, including the motor dysfunctions caused by neurological diseases such as Parkinson’s disease or hemiplegia after stroke. Monitoring changes and deterioration in gait patterns and balance is crucial for activating rehabilitation treatments and preventing serious consequences. This work presents a Kinect-based solution, suitable for domestic contexts, for assessing gait and balance in individuals at risk of falling. The system captures body movements during home acquisition sessions scheduled by clinicians at definite times of the day and automatically estimates specific functional parameters to objectively characterize the subjects’ performance. The system includes a graphical user interface designed to ensure usability in unsupervised contexts: the human-computer interaction mainly relies on natural body movements to support the self-management of the system, if the motor conditions allow it. This work presents the system’s features and facilities, and the preliminary results on healthy volunteers’ trials