Using the Microsoft Kinect to assess human bimanual coordination

Optical marker-based systems are the gold-standard for capturing three-dimensional (3D) human kinematics. However, these systems have various drawbacks including time consuming marker placement, soft tissue movement artifact, and are prohibitively expensive and non-portable. The Microsoft Kinect is an inexpensive, portable, depth camera that can be used to capture 3D human movement kinematics. Numerous investigations have assessed the Kinect\u27s ability to capture postural control and gait, but to date, no study has evaluated it\u27s capabilities for measuring spatiotemporal coordination. In order to investigate human coordination and coordination stability with the Kinect, a well-studied bimanual coordination paradigm (Kelso, 1984, Kelso; Scholz, & Schöner, 1986) was adapted. ^ Nineteen participants performed ten trials of coordinated hand movements in either in-phase or anti-phase patterns of coordination to the beat of a metronome which was incrementally sped up and slowed down. Continuous relative phase (CRP) and the standard deviation of CRP were used to assess coordination and coordination stability, respectively.^ Data from the Kinect were compared to a Vicon motion capture system using a mixed-model, repeated measures analysis of variance and intraclass correlation coefficients (2,1) (ICC(2,1)).^ Kinect significantly underestimated CRP for the the anti-phase coordination pattern (p \u3c.0001) and overestimated the in-phase pattern (p\u3c.0001). However, a high ICC value (r=.097) was found between the systems. For the standard deviation of CRP, the Kinect exhibited significantly higher variability than the Vicon (p \u3c .0001) but was able to distinguish significant differences between patterns of coordination with anti-phase variability being higher than in-phase (p \u3c .0001). Additionally, the Kinect was unable to accurately capture the structure of coordination stability for the anti-phase pattern. Finally, agreement was found between systems using the ICC (r=.37).^ In conclusion, the Kinect was unable to accurately capture mean CRP. However, the high ICC between the two systems is promising and the Kinect was able to distinguish between the coordination stability of in-phase and anti-phase coordination. However, the structure of variability as movement speed increased was dissimilar to the Vicon, particularly for the anti-phase pattern. Some aspects of coordination are nicely captured by the Kinect while others are not. Detecting differences between bimanual coordination patterns and the stability of those patterns can be achieved using the Kinect. However, researchers interested in the structure of coordination stability should exercise caution since poor agreement was found between systems

Using perceptive computing in multiple sclerosis - the Short Maximum Speed Walk test

BACKGROUND: We investigated the applicability and feasibility of perceptive computing assisted gait analysis in multiple sclerosis (MS) patients using Microsoft Kinect. To detect the maximum walking speed and the degree of spatial sway, we established a computerized and observer-independent measure, which we named Short Maximum Speed Walk (SMSW), and compared it to established clinical measures of gait disability in MS, namely the Expanded Disability Status Scale (EDSS) and the Timed 25-Foot Walk (T25FW). METHODS: Cross-sectional study of 22 MS patients (age mean +/- SD 43 +/- 9 years, 13 female) and 22 age and gender matched healthy control subjects (HC) (age 37 +/- 11 years, 13 female). The disability level of each MS patient was graded using the EDSS (median 3.0, range 0.0-6.0). All subjects then performed the SMSW and the Timed 25-Foot Walk (T25FW). The SMSW comprised five gait parameters, which together assessed average walking speed and gait stability in different dimensions (left/right, up/down and 3D deviation). RESULTS: SMSW average walking speed was slower in MS patients (1.6 +/- 0.3 m/sec) than in HC (1.8 +/- 0.4 m/sec) (p = 0.005) and correlated well with EDSS (Spearman's Rho 0.676, p < 0.001). Furthermore, SMSW revealed higher left/right deviation in MS patients compared to HC. SMSW showed high recognition quality and retest-reliability (covariance 0.13 m/sec, ICC 0.965, p < 0.001). There was a significant correlation between SMSW average walking speed and T25FW (Pearson's R = -0.447, p = 0.042). CONCLUSION: Our data suggest that ambulation tests using Microsoft Kinect are feasible, well tolerated and can detect clinical gait disturbances in patients with MS. The retest-reliability was on par with the T25FW

Development of centroid based metrics to provide personalized monitoring and assessment of gait

A methodology is presented to characterize a subject's ability to ambulate using various metrics generated from the movement of the subject's centroid as detected using an inexpensive depth camera system. Metrics have been developed focusing on three major categories of motion. The first, and most basic, class measures characteristics of movement in the direction of motion, laterally, and vertically. The second class focuses on measuring the entropy that exists in the subject's walk. The third class uses periodicity in the subject's motion to deduce temporal gait parameters including stride length, and step times on the left and right side. As each patient is unique with different histories, disease progression, and overall state, metrics and the associated analysis approaches integrate a personalized approach to selecting and using metrics. These stride time, stride length, and average speed metrics were then validated against both the Vicon[superscript TM] system and an established reference algorithm. From these metrics, a set of methodologies were developed to study short and long-term effects of therapies, significant changes in metrics due to clinical events, as well as the ability to predict potential clinical events by identifying outliers in long term trends. These metrics and my analysis approach were then tested using a group of subjects undergoing therapy using strategic weighted vests. The ability of the metrics to show changes in the subject's ambulation when the vest is either put on, or taken off was examined. Results show sufficient sensitivity to detect changes when the vest is donned and doffed. Interestingly, results also show that the effects of the vest are not seen immediately, but over 2-4 hours following donning. Results also demonstrate the ability, using the window size, to focus on the time required for the effects of each metric to change. Lastly, results show distinct results for each individual subject. Additional studies were also done using subjects not undergoing the vest therapy to identify trends and outliers as portents of clinical events. Results show the ability to identify potential clinical events by identifying outliers in long term trends. Again, results are improved if the metrics used in the analysis are chosen specific to each subject. The metrics are also compared against existing Fall Risk Assessments to validate their potential usefulness in monitoring subjects for changing risk of falls. While results show strong correlation to some FRA's, not every subject has the same relationships between metrics and FRA's suggesting a unique "fingerprint" of metrics associated with a subject's condition. Lastly, the performance of these metrics was evaluated against a reference algorithm using in home data as well as against in home data into which a simulated obstruction was placed. In both cases, greater than 90% of the walks could produce a valid set of metrics and the simulated obstruction resulted in no significant changes in the examined metrics. The metrics and methodologies presented here show a significant improvement in performance, a wider diversity of measurements, as well as the ability to measure both short term and long-term trends than currently existing approaches.Includes biblographical reference

Deriving information from spatial sampling floor-based personnel detection system

Field of study: Electrical & computer engineering.Dr. Harry W. Tyrer, Thesis Supervisor.Includes vita."May 2017."Research has shown and identified a clear link between human gait characteristics and different medical conditions. Therefore, a change in certain gait parameters may be predictive of future falls and adverse events in older adults such as physical functional decline and fall risks. We describe a system that is unobtrusive and continuously monitors the gait during daily activities of elderly people. The early assessment of gait decline will benefit the senior by providing an indication of the risk of falls. We developed a low cost floor-based personnel detection system; we call a smart carpet, which consists of a sensor pad placed under a carpet; the electronics reads walking activity. The smart carpet systems is used as a component of an automated health monitoring system, which helps enable independent living for elderly people and provide a practical environment that improves quality of life, reduces healthcare costs and promotes independence. In this dissertation, we extended the functionalities of the smart carpet to improve its ability to detect falls, estimate gait parameters and compared it to GAITRite system. We counted number of people walking on the carpet in order to distinguish the plurality of people from fall event. Additionally we studied the characteristics and the behavior of the sensor's scavenged signal. Results showed that our system detects falls, using computational intelligence techniques, with 96.2% accuracy and 81% sensitivity and 97.8% specificity. The system reliably estimates the gait parameters; walking speed, stride length and stride time with percentage errors of 1.43%, -4.32%, and -5.73% respectively. Our system can count the number of people on the carpet with high accuracy, and we ran tests with up to four people. We were able to use computational features of the generated waveform, by extracting the Mel Frequency Cepstral Coefficients (MFCC), and using formal computation intelligence to distinguish different people with an average accuracy of 82%, given that the experiments were performed within the same day.Includes bibliographical references

Walking Gait Measurement and Gait Parameters Extraction

A fourth generation walking gait measurement device has been designed to capture and analyze detailed gait and stride metrics which eventually provides a Fall-Risk Assessment score. Specifically, the device has been modified to fit the residential environment and the elderly consumers which is low-cost, user-friendly, and portable. The gait parameters would be obtained by the on-board gait analysis protocol. Through gait parameters people’s falling risk can then be calculated so that people can be alerted to take precautionary measures before falling. Overall, the device has been made and demonstrated having better performance than its previous generations. The on-board gait analyzing program executed slower than the computer version program but has the same accuracy. However, the overall performance is better than transforming data from the measurement device to a computer manually

State of the art of audio- and video based solutions for AAL

Working Group 3. Audio- and Video-based AAL ApplicationsIt is a matter of fact that Europe is facing more and more crucial challenges regarding health and social care due to the demographic change and the current economic context. The recent COVID-19 pandemic has stressed this situation even further, thus highlighting the need for taking action. Active and Assisted Living (AAL) technologies come as a viable approach to help facing these challenges, thanks to the high potential they have in enabling remote care and support. Broadly speaking, AAL can be referred to as the use of innovative and advanced Information and Communication Technologies to create supportive, inclusive and empowering applications and environments that enable older, impaired or frail people to live independently and stay active longer in society. AAL capitalizes on the growing pervasiveness and effectiveness of sensing and computing facilities to supply the persons in need with smart assistance, by responding to their necessities of autonomy, independence, comfort, security and safety. The application scenarios addressed by AAL are complex, due to the inherent heterogeneity of the end-user population, their living arrangements, and their physical conditions or impairment. Despite aiming at diverse goals, AAL systems should share some common characteristics. They are designed to provide support in daily life in an invisible, unobtrusive and user-friendly manner. Moreover, they are conceived to be intelligent, to be able to learn and adapt to the requirements and requests of the assisted people, and to synchronise with their specific needs. Nevertheless, to ensure the uptake of AAL in society, potential users must be willing to use AAL applications and to integrate them in their daily environments and lives. In this respect, video- and audio-based AAL applications have several advantages, in terms of unobtrusiveness and information richness. Indeed, cameras and microphones are far less obtrusive with respect to the hindrance other wearable sensors may cause to one’s activities. In addition, a single camera placed in a room can record most of the activities performed in the room, thus replacing many other non-visual sensors. Currently, video-based applications are effective in recognising and monitoring the activities, the movements, and the overall conditions of the assisted individuals as well as to assess their vital parameters (e.g., heart rate, respiratory rate). Similarly, audio sensors have the potential to become one of the most important modalities for interaction with AAL systems, as they can have a large range of sensing, do not require physical presence at a particular location and are physically intangible. Moreover, relevant information about individuals’ activities and health status can derive from processing audio signals (e.g., speech recordings). Nevertheless, as the other side of the coin, cameras and microphones are often perceived as the most intrusive technologies from the viewpoint of the privacy of the monitored individuals. This is due to the richness of the information these technologies convey and the intimate setting where they may be deployed. Solutions able to ensure privacy preservation by context and by design, as well as to ensure high legal and ethical standards are in high demand. After the review of the current state of play and the discussion in GoodBrother, we may claim that the first solutions in this direction are starting to appear in the literature. A multidisciplinary 4 debate among experts and stakeholders is paving the way towards AAL ensuring ergonomics, usability, acceptance and privacy preservation. The DIANA, PAAL, and VisuAAL projects are examples of this fresh approach. This report provides the reader with a review of the most recent advances in audio- and video-based monitoring technologies for AAL. It has been drafted as a collective effort of WG3 to supply an introduction to AAL, its evolution over time and its main functional and technological underpinnings. In this respect, the report contributes to the field with the outline of a new generation of ethical-aware AAL technologies and a proposal for a novel comprehensive taxonomy of AAL systems and applications. Moreover, the report allows non-technical readers to gather an overview of the main components of an AAL system and how these function and interact with the end-users. The report illustrates the state of the art of the most successful AAL applications and functions based on audio and video data, namely (i) lifelogging and self-monitoring, (ii) remote monitoring of vital signs, (iii) emotional state recognition, (iv) food intake monitoring, activity and behaviour recognition, (v) activity and personal assistance, (vi) gesture recognition, (vii) fall detection and prevention, (viii) mobility assessment and frailty recognition, and (ix) cognitive and motor rehabilitation. For these application scenarios, the report illustrates the state of play in terms of scientific advances, available products and research project. The open challenges are also highlighted. The report ends with an overview of the challenges, the hindrances and the opportunities posed by the uptake in real world settings of AAL technologies. In this respect, the report illustrates the current procedural and technological approaches to cope with acceptability, usability and trust in the AAL technology, by surveying strategies and approaches to co-design, to privacy preservation in video and audio data, to transparency and explainability in data processing, and to data transmission and communication. User acceptance and ethical considerations are also debated. Finally, the potentials coming from the silver economy are overviewed.publishedVersio

Comparing Microsoft Kinect and Observational Gait Analysis in Assessing Gait Parameters of Apparently Healthy Adults

Objectives: Although the Microsoft Kinect has compelling potential for gait analysis in medicine, data available to compare it with observational gait analysis (OGA) is scarce. This study compared the Microsoft Kinect and the OGA in assessing the gait parameters of apparently healthy adults. Methods: Ninety-seven apparently healthy young male adults participated in this comparative study. First, the participant’s age, height, weight, and body mass index were obtained. Afterward, gait parameters involving the number of steps, cadence, stride length, and step length were assessed concurrently following OGA standard procedures and the Microsoft Kinect during a 6-m walk down the hallway. The obtained data were analyzed using descriptive and inferential statistics. The significance level was set at P<0.05. Results: The Mean±SD walk time, steps, cadence, velocity, and stride length were 8.07±1.39 s, 14.0±2.96 counts, 72.9±11.9 steps/min, 0.8±0.13 m/s, and 0.77±0.13m, respectively. Step length was significantly higher (P<0.05) with Microsoft Kinect than OGA, whereas stride length and walk speed values were significantly (P<0.05) lower with Microsoft Kinect. A moderate but significant (P=0.001) positive correlation existed between Microsoft Kinect and OGA regarding walk speed. In contrast, regarding the step length, a weak but significant (P<0.05) positive correlation was found between Microsoft Kinect and OGA. Discussion: Step length values of Microsoft Kinect were significantly higher than OGA values, whereas stride length and walk speed values of Microsoft Kinect were significantly lower than OGA values. Walk speed and step length measured by Microsoft Kinect and OGA were positively correlated

A review of computer vision-based approaches for physical rehabilitation and assessment

The computer vision community has extensively researched the area of human motion analysis, which primarily focuses on pose estimation, activity recognition, pose or gesture recognition and so on. However for many applications, like monitoring of functional rehabilitation of patients with musculo skeletal or physical impairments, the requirement is to comparatively evaluate human motion. In this survey, we capture important literature on vision-based monitoring and physical rehabilitation that focuses on comparative evaluation of human motion during the past two decades and discuss the state of current research in this area. Unlike other reviews in this area, which are written from a clinical objective, this article presents research in this area from a computer vision application perspective. We propose our own taxonomy of computer vision-based rehabilitation and assessment research which are further divided into sub-categories to capture novelties of each research. The review discusses the challenges of this domain due to the wide ranging human motion abnormalities and difficulty in automatically assessing those abnormalities. Finally, suggestions on the future direction of research are offered

Machine Learning-based Detection of Compensatory Balance Responses and Environmental Fall Risks Using Wearable Sensors

Falls are the leading cause of fatal and non-fatal injuries among seniors worldwide, with serious and costly consequences. Compensatory balance responses (CBRs) are reactions to recover stability following a loss of balance, potentially resulting in a fall if sufficient recovery mechanisms are not activated. While performance of CBRs are demonstrated risk factors for falls in seniors, the frequency, type, and underlying cause of these incidents occurring in everyday life have not been well investigated. This study was spawned from the lack of research on development of fall risk assessment methods that can be used for continuous and long-term mobility monitoring of the geri- atric population, during activities of daily living, and in their dwellings. Wearable sensor systems (WSS) offer a promising approach for continuous real-time detection of gait and balance behavior to assess the risk of falling during activities of daily living. To detect CBRs, we record movement signals (e.g. acceleration) and activity patterns of four muscles involving in maintaining balance using wearable inertial measurement units (IMUs) and surface electromyography (sEMG) sensors. To develop more robust detection methods, we investigate machine learning approaches (e.g., support vector machines, neural networks) and successfully detect lateral CBRs, during normal gait with accuracies of 92.4% and 98.1% using sEMG and IMU signals, respectively. Moreover, to detect environmental fall-related hazards that are associated with CBRs, and affect balance control behavior of seniors, we employ an egocentric mobile vision system mounted on participants chest. Two algorithms (e.g. Gabor Barcodes and Convolutional Neural Networks) are developed. Our vision-based method detects 17 different classes of environmental risk factors (e.g., stairs, ramps, curbs) with 88.5% accuracy. To the best of the authors knowledge, this study is the first to develop and evaluate an automated vision-based method for fall hazard detection

Wearables for Movement Analysis in Healthcare

Quantitative movement analysis is widely used in clinical practice and research to investigate movement disorders objectively and in a complete way. Conventionally, body segment kinematic and kinetic parameters are measured in gait laboratories using marker-based optoelectronic systems, force plates, and electromyographic systems. Although movement analyses are considered accurate, the availability of specific laboratories, high costs, and dependency on trained users sometimes limit its use in clinical practice. A variety of compact wearable sensors are available today and have allowed researchers and clinicians to pursue applications in which individuals are monitored in their homes and in community settings within different fields of study, such movement analysis. Wearable sensors may thus contribute to the implementation of quantitative movement analyses even during out-patient use to reduce evaluation times and to provide objective, quantifiable data on the patients’ capabilities, unobtrusively and continuously, for clinical purposes