Recognition of Postures and Freezing of Gait in Parkinson’s Disease Patients Using Microsoft Kinect Sensor

Freezing of Gait (FOG) is a disabling symptom and movement disorder, typically associated with the latter stages of Parkinson’s disease. In this paper, we propose a novel approach for real-time FOG, tremor monitoring and fall detection, consisting of a 3D camera sensor based on the Microsoft Kinect architecture. The system is capable of recognizing freezing episodes (FOG) in a standstill state, tremors and fall incidents, commonly seen in Parkinson’s disease patients. In case of an incident, it automatically alerts relatives and healthcare providers. The system was tested on seven simulated subjects in 12 events indicating that the design was able to detect 99% of the falling incidents, 91% of tremor and 92% of the freezing of gait episodes with an average latency of 300 milliseconds. The performance of the system can be further improved with the deployment of the recently released version of Kinect, capable of providing even higher levels of accuracy

Using 3D sensing and projecting technology to improve the mobility of Parkinson's disease patients

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University LondonParkinson’s is a neurological condition in which parts of the brain responsible for movements becomes incapacitated over time due to the abnormal dopamine equilibrium. Freezing of Gait (FOG) is one of the main Parkinson’s Disease (PD) symptoms that affects patients not only physically but also psychologically as it prevents them from fulfilling simple tasks such as standing up or walking. Different auditory and visual cues have been proven to be very effective in improving the mobility of People with Parkinson’s (PwP). Nonetheless, many of the available methods require user intervention or devices to be worn, charged, etc. to activate the cues. This research suggests a system that can provide an unobtrusive facility to detect FOG and falling in PwP as well as monitoring and improving their mobility using laser-based visual cues casted by an automated laser system. It proposes a new indoor method for casting a set of two parallel laser lines as a dynamic visual cue in front of a subject’s feet based on the subject’s head direction and 3D location in a room. The proposed system controls the movement of a set of pan/tilt servo motors and laser pointers using a microcontroller based on the real-time skeletal information acquired from a Kinect v2 sensor. A Graphical User Interface (GUI) is created that enables users to control and adjust the settings based on the user preferences. The system was tested and trained by 12 healthy participants and reviewed by 15 PwP who suffer from frequent FOG episodes. The results showed the possibility of employing the system as an indoor and on-demand visual cue system for PwP that does not rely on the subject’s input or introduce any additional complexities to operate. Despite limitations regarding its outdoor use, feedback was very positive in terms of domestic usability and convenience, where 12/15 PwP showed interest in installing and using the system at their homes

Using Kinect v2 to Control a Laser Visual Cue System to Improve the Mobility during Freezing of Gait in Parkinson’s Disease

Different auditory and visual cues have been proven to be very effective in improving the mobility of People with Parkinson’s (PwP). Nonetheless, many of the available methods require user intervention, etc. to activate the cues. Moreover, once activated, these systems would provide cues continuously regardless of the patient’s needs. This research proposes a new indoor method for casting dynamic/automatic visual cues for PwP based on their head direction and location in a room. The proposed system controls the behavior of a set of pan/tilt servo motors and laser pointers, based on the real-time skeletal information acquired from a Kinect v2 sensor. This produces an automatically adjusting set of laser lines that can always be in front of the patient as a guideline for where the next footstep would be placed. A user interface was also created that enables users to control and adjust the settings based on the preferences. The aim of this research was to provide PwP with an unobtrusive/automatic indoor system for improving their mobility during a Freezing of Gait (FOG) incident. The results showed the possibility of employing such system, which does not rely on the subject’s input nor does it introduce any additional complexities to operate

Sistema para análise automatizada de movimento durante a marcha usando uma câmara RGB-D

Nowadays it is still common in clinical practice to assess the gait (or way of walking) of a given subject through the visual observation and use of a rating scale, which is a subjective approach. However, sensors including RGB-D cameras, such as the Microsoft Kinect, can be used to obtain quantitative information that allows performing gait analysis in a more objective way. The quantitative gait analysis results can be very useful for example to support the clinical assessment of patients with diseases that can affect their gait, such as Parkinson’s disease. The main motivation of this thesis was thus to provide support to gait assessment, by allowing to carry out quantitative gait analysis in an automated way. This objective was achieved by using 3-D data, provided by a single RGB-D camera, to automatically select the data corresponding to walking and then detect the gait cycles performed by the subject while walking. For each detected gait cycle, we obtain several gait parameters, which are used together with anthropometric measures to automatically identify the subject being assessed. The automated gait data selection relies on machine learning techniques to recognize three different activities (walking, standing, and marching), as well as two different positions of the subject in relation to the camera (facing the camera and facing away from it). For gait cycle detection, we developed an algorithm that estimates the instants corresponding to given gait events. The subject identification based on gait is enabled by a solution that was also developed by relying on machine learning. The developed solutions were integrated into a system for automated gait analysis, which we found to be a viable alternative to gold standard systems for obtaining several spatiotemporal and some kinematic gait parameters. Furthermore, the system is suitable for use in clinical environments, as well as ambulatory scenarios, since it relies on a single markerless RGB-D camera that is less expensive, more portable, less intrusive and easier to set up, when compared with the gold standard systems (multiple cameras and several markers attached to the subject’s body).Atualmente ainda é comum na prática clínica avaliar a marcha (ou o modo de andar) de uma certa pessoa através da observação visual e utilização de uma escala de classificação, o que é uma abordagem subjetiva. No entanto, existem sensores incluindo câmaras RGB-D, como a Microsoft Kinect, que podem ser usados para obter informação quantitativa que permite realizar a análise da marcha de um modo mais objetivo. Os resultados quantitativos da análise da marcha podem ser muito úteis, por exemplo, para apoiar a avaliação clínica de pessoas com doenças que podem afetar a sua marcha, como a doença de Parkinson. Assim, a principal motivação desta tese foi fornecer apoio à avaliação da marcha, permitindo realizar a análise quantitativa da marcha de forma automatizada. Este objetivo foi atingido usando dados em 3-D, fornecidos por uma única câmara RGB-D, para automaticamente selecionar os dados correspondentes a andar e, em seguida, detetar os ciclos de marcha executados pelo sujeito durante a marcha. Para cada ciclo de marcha identificado, obtemos vários parâmetros de marcha, que são usados em conjunto com medidas antropométricas para identificar automaticamente o sujeito que está a ser avaliado. A seleção automatizada de dados de marcha usa técnicas de aprendizagem máquina para reconhecer três atividades diferentes (andar, estar parado em pé e marchar), bem como duas posições diferentes do sujeito em relação à câmara (de frente para a câmara e de costas para ela). Para a deteção dos ciclos da marcha, desenvolvemos um algoritmo que estima os instantes correspondentes a determinados eventos da marcha. A identificação do sujeito com base na sua marcha é realizada usando uma solução que também foi desenvolvida com base em aprendizagem máquina. As soluções desenvolvidas foram integradas num sistema de análise automatizada de marcha, que demonstrámos ser uma alternativa viável a sistemas padrão de referência para obter vários parâmetros de marcha espácio-temporais e alguns parâmetros angulares. Além disso, o sistema é adequado para uso em ambientes clínicos, bem como em cenários ambulatórios, pois depende de apenas de uma câmara RGB-D que não usa marcadores e é menos dispendiosa, mais portátil, menos intrusiva e mais fácil de configurar, quando comparada com os sistemas padrão de referência (múltiplas câmaras e vários marcadores colocados no corpo do sujeito).Programa Doutoral em Informátic

Gait analysis in neurological populations: Progression in the use of wearables

Gait assessment is an essential tool for clinical applications not only to diagnose different neurological conditions but also to monitor disease progression as it contributes to the understanding of underlying deficits. There are established methods and models for data collection and interpretation of gait assessment within different pathologies. This narrative review aims to depict the evolution of gait assessment from observation and rating scales to wearable sensors and laboratory technologies, and provide possible future directions. In this context, we first present an extensive review of current clinical outcomes and gait models. Then, we demonstrate commercially available wearable technologies with their technical capabilities along with their use in gait assessment studies for various neurological conditions. In the next sections, a descriptive knowledge for existing inertial based algorithms and a sign based guide that shows the outcomes of previous neurological gait assessment studies are presented. Finally, we state a discussion for the use of wearables in gait assessment and speculate the possible research directions by revealing the limitations and knowledge gaps in the literature

Evaluación del movimiento en pacientes con parkinson utilizando técnicas computacionales para la automatización del protocolo UPDRS mediante el Kinect

La enfermedad de Párkinson EP, es una patología de carácter neurodegenerativo, la cual afecta los aspectos motores y cognitivos en una persona. La ausencia de dopamina en las neuronas que se localizan en la sustancia negra a nivel cerebral genera alteraciones en el monitoreo de los aspectos motores del cuerpo. Esta enfermedad presenta una sintomatología como temblor en reposo, bradicinesia, rigidez y alteraciones en la marcha. A su vez, la EP desarrolla un cuadro de evolución temporal. Por esta razón es importante la puesta en ejecución de tratamientos clínicos para supervisar el estado de la enfermedad y observar su evolución. Actualmente existe un protocolo que evalúa características cognitivas y ejercicios físicos en pacientes con EP, denominado la Escala unificada de clasificación de la enfermedad de Parkinson UPDRS, desarrolladas por un especialista en neurología. Sin embargo, es una prueba subjetiva a la hora de dar un diagnóstico. En la literatura se encuentran propuestas para obtener una estimación de características de forma cuantitativa, pese a esto se tiene la necesidad de reconocer ciertos ejercicios que son específicos de la EP y que no se han automatizados en el protocolo de aspectos motores del UPDRS. Por esta razón se propone realizar la captura por computador de características cuantitativas en articulaciones, analizando los movimientos especificados en el protocolo UPDRS

Development of Markerless Systems for Automatic Analysis of Movements and Facial Expressions: Applications in Neurophysiology

This project is focused on the development of markerless methods for studying facial expressions and movements in neurology, focusing on Parkinson’s disease (PD) and disorders of consciousness (DOC). PD is a neurodegenerative illness that affects around 2% of the population over 65 years old. Impairments of voice/speech are among the main signs of PD. This set of impairments is called hypokinetic dysarthria, because of the reduced range of movements involved in speech. This reduction can be visible also in other facial muscles, leading to a hypomimia. Despite the high percentage of patients that suffer from dysarthria and hypomimia, only a few of them undergo speech therapy with the aim to improve the dynamic of articulatory/facial movements. The main reason is the lack of low cost methodologies that could be implemented at home. DOC after coma are Vegetative State (VS), characterized by the absence of self-awareness and awareness of the environment, and Minimally Conscious State (MCS), in which certain behaviors are sufficiently reproducible to be distinguished from reflex responses. The differential diagnosis between VS and MCS can be hard and prone to a high rate of misdiagnosis (~40%). This differential diagnosis is mainly based on neuro-behavioral scales. A key role to plan the rehabilitation in DOC patients is played by the first diagnosis after coma. In fact, MCS patients are more prone to a consciousness recovery than VS patients. Concerning PD the aim is the development of contactless systems that could be used to study symptoms related to speech and facial movements/expressions. The methods proposed here, based on acoustical analysis and video processing techniques could support patients during speech therapy also at home. Concerning DOC patients the project is focused on the assessment of reflex and cognitive responses to standardized stimuli. This would allow objectifying the perceptual analysis performed by clinicians

An automatic wearable multi-sensor based gait analysis system for older adults.

Gait abnormalities in older adults are very common in clinical practice. They lead to serious adverse consequences such as falls and injury resulting in increased care cost. There is therefore a national imperative to address this challenge. Currently gait assessment is done using standardized clinical tools dependent on subjective evaluation. More objective gold standard methods (motion capture systems such as Qualisys and Vicon) to analyse gait rely on access to expensive complex equipment based in gait laboratories. These are not widely available for several reasons including a scarcity of equipment, need for technical staff, need for patients to attend in person, complicated time consuming procedures and overall expense. To broaden the use of accurate quantitative gait monitoring and assessment, the major goal of this thesis is to develop an affordable automatic gait analysis system that will provide comprehensive gait information and allow use in clinic or at home. It will also be able to quantify and visualize gait parameters, identify gait variables and changes, monitor abnormal gait patterns of older people in order to reduce the potential for falling and support falls risk management. A research program based on conducting experiments on volunteers is developed in collaboration with other researchers in Bournemouth University, The Royal Bournemouth Hospital and care homes. This thesis consists of five different studies toward addressing our major goal. Firstly, a study on the effects on sensor output from an Inertial Measurement Unit (IMU) attached to different anatomical foot locations. Placing an IMU over the bony prominence of the first cuboid bone is the best place as it delivers the most accurate data. Secondly, an automatic gait feature extraction method for analysing spatiotemporal gait features which shows that it is possible to extract gait features automatically outside of a gait laboratory. Thirdly, user friendly and easy to interpret visualization approaches are proposed to demonstrate real time spatiotemporal gait information. Four proposed approaches have the potential of helping professionals detect and interpret gait asymmetry. Fourthly, a validation study of spatiotemporal IMU extracted features compared with gold standard Motion Capture System and Treadmill measurements in young and older adults is conducted. The results obtained from three experimental conditions demonstrate that our IMU gait extracted features are highly valid for spatiotemporal gait variables in young and older adults. In the last study, an evaluation system using Procrustes and Euclidean distance matrix analysis is proposed to provide a comprehensive interpretation of shape and form differences between individual gaits. The results show that older gaits are distinguishable from young gaits. A pictorial and numerical system is proposed which indicates whether the assessed gait is normal or abnormal depending on their total feature values. This offers several advantages: 1) it is user friendly and is easy to set up and implement; 2) it does not require complex equipment with segmentation of body parts; 3) it is relatively inexpensive and therefore increases its affordability decreasing health inequality; and 4) its versatility increases its usability at home supporting inclusivity of patients who are home bound. A digital transformation strategy framework is proposed where stakeholders such as patients, health care professionals and industry partners can collaborate through development of new technologies, value creation, structural change, affordability and sustainability to improve the diagnosis and treatment of gait abnormalities