Video-Based Hand Movement Analysis of Parkinson Patients before and after Medication Using High-Frame-Rate Videos and MediaPipe

Tremor is one of the common symptoms of Parkinson’s disease (PD). Thanks to the recent evolution of digital technologies, monitoring of PD patients’ hand movements employing contactless methods gained momentum. Objective: We aimed to quantitatively assess hand movements in patients suffering from PD using the artificial intelligence (AI)-based hand-tracking technologies of MediaPipe. Method: High-frame-rate videos and accelerometer data were recorded from 11 PD patients, two of whom showed classical Parkinsonian-type tremor. In the OFF-state and 30 Minutes after taking their standard oral medication (ON-state), video recordings were obtained. First, we investigated the frequency and amplitude relationship between the video and accelerometer data. Then, we focused on quantifying the effect of taking standard oral treatments. Results: The data extracted from the video correlated well with the accelerometer-based measurement system. Our video-based approach identified the tremor frequency with a small error rate (mean absolute error 0.229 (±0.174) Hz) and an amplitude with a high correlation. The frequency and amplitude of the hand movement before and after medication in PD patients undergoing medication differ. PD Patients experienced a decrease in the mean value for frequency from 2.012 (±1.385) Hz to 1.526 (±1.007) Hz and in the mean value for amplitude from 8.167 (±15.687) a.u. to 4.033 (±5.671) a.u. Conclusions: Our work achieved an automatic estimation of the movement frequency, including the tremor frequency with a low error rate, and to the best of our knowledge, this is the first paper that presents automated tremor analysis before/after medication in PD, in particular using high-frame-rate video data

Video Acceleration Magnification

The ability to amplify or reduce subtle image changes over time is useful in contexts such as video editing, medical video analysis, product quality control and sports. In these contexts there is often large motion present which severely distorts current video amplification methods that magnify change linearly. In this work we propose a method to cope with large motions while still magnifying small changes. We make the following two observations: i) large motions are linear on the temporal scale of the small changes; ii) small changes deviate from this linearity. We ignore linear motion and propose to magnify acceleration. Our method is pure Eulerian and does not require any optical flow, temporal alignment or region annotations. We link temporal second-order derivative filtering to spatial acceleration magnification. We apply our method to moving objects where we show motion magnification and color magnification. We provide quantitative as well as qualitative evidence for our method while comparing to the state-of-the-art.Comment: Accepted paper at CVPR 2017. Project webpage: http://acceleration-magnification.github.io

The discerning eye of computer vision: can it measure Parkinson's finger tap bradykinesia?

Objective: The worldwide prevalence of Parkinson's disease is increasing. There is urgent need for new tools to objectively measure the condition. Existing methods to record the cardinal motor feature of the condition, bradykinesia, using wearable sensors or smartphone apps have not reached large-scale, routine use. We evaluate new computer vision (artificial intelligence) technology, DeepLabCut, as a contactless method to quantify measures related to Parkinson's bradykinesia from smartphone videos of finger tapping. Methods: Standard smartphone video recordings of 133 hands performing finger tapping (39 idiopathic Parkinson's patients and 30 controls) were tracked on a frame-by-frame basis with DeepLabCut. Objective computer measures of tapping speed, amplitude and rhythm were correlated with clinical ratings made by 22 movement disorder neurologists using the Modified Bradykinesia Rating Scale (MBRS) and Movement Disorder Society revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS). Results: DeepLabCut reliably tracked and measured finger tapping in standard smartphone video. Computer measures correlated well with clinical ratings of bradykinesia (Spearman coefficients): -0.74 speed, 0.66 amplitude, -0.65 rhythm for MBRS; -0.56 speed, 0.61 amplitude, -0.50 rhythm for MDS-UPDRS; -0.69 combined for MDS-UPDRS. All p Conclusion: New computer vision software, DeepLabCut, can quantify three measures related to Parkinson's bradykinesia from smartphone videos of finger tapping. Objective 'contactless' measures of standard clinical examinations were not previously possible with wearable sensors (accelerometers, gyroscopes, infrared markers). DeepLabCut requires only conventional video recording of clinical examination and is entirely 'contactless'. This next generation technology holds potential for Parkinson's and other neurological disorders with altered movements

Pose-based Tremor Classification for Parkinson's Disease Diagnosis from Video

Parkinson's disease (PD) is a progressive neurodegenerative disorder that results in a variety of motor dysfunction symptoms, including tremors, bradykinesia, rigidity and postural instability. The diagnosis of PD mainly relies on clinical experience rather than a definite medical test, and the diagnostic accuracy is only about 73-84% since it is challenged by the subjective opinions or experiences of different medical experts. Therefore, an efficient and interpretable automatic PD diagnosis system is valuable for supporting clinicians with more robust diagnostic decision-making. To this end, we propose to classify Parkinson's tremor since it is one of the most predominant symptoms of PD with strong generalizability. Different from other computer-aided time and resource-consuming Parkinson's Tremor (PT) classification systems that rely on wearable sensors, we propose SPAPNet, which only requires consumer-grade non-intrusive video recording of camera-facing human movements as input to provide undiagnosed patients with low-cost PT classification results as a PD warning sign. For the first time, we propose to use a novel attention module with a lightweight pyramidal channel-squeezing-fusion architecture to extract relevant PT information and filter the noise efficiently. This design aids in improving both classification performance and system interpretability. Experimental results show that our system outperforms state-of-the-arts by achieving a balanced accuracy of 90.9% and an F1-score of 90.6% in classifying PT with the non-PT class.Comment: MICCAI 202

Neurological Tremor: Sensors, Signal Processing and Emerging Applications

Neurological tremor is the most common movement disorder, affecting more than 4% of elderly people. Tremor is a non linear and non stationary phenomenon, which is increasingly recognized. The issue of selection of sensors is central in the characterization of tremor. This paper reviews the state-of-the-art instrumentation and methods of signal processing for tremor occurring in humans. We describe the advantages and disadvantages of the most commonly used sensors, as well as the emerging wearable sensors being developed to assess tremor instantaneously. We discuss the current limitations and the future applications such as the integration of tremor sensors in BCIs (brain-computer interfaces) and the need for sensor fusion approaches for wearable solutions

Assigning UPDRS Scores in the Leg Agility Task of Parkinsonians: Can It Be Done through BSN-based Kinematic Variables?

In this paper, by characterizing the Leg Agility (LA) task, which contributes to the evaluation of the degree of severity of the Parkinson's Disease (PD), through kinematic variables (including the angular amplitude and speed of thighs' motion), we investigate the link between these variables and Unified Parkinson's Disease Rating Scale (UPDRS) scores. Our investigation relies on the use of a few body-worn wireless inertial nodes and represents a first step in the design of a portable system, amenable to be integrated in Internet of Things (IoT) scenarios, for automatic detection of the degree of severity (in terms of UPDRS score) of PD. The experimental investigation is carried out considering 24 PD patients.Comment: 10 page

Vision-Based Assessment of Parkinsonism and Levodopa-Induced Dyskinesia with Deep Learning Pose Estimation

Objective: To apply deep learning pose estimation algorithms for vision-based assessment of parkinsonism and levodopa-induced dyskinesia (LID). Methods: Nine participants with Parkinson's disease (PD) and LID completed a levodopa infusion protocol, where symptoms were assessed at regular intervals using the Unified Dyskinesia Rating Scale (UDysRS) and Unified Parkinson's Disease Rating Scale (UPDRS). A state-of-the-art deep learning pose estimation method was used to extract movement trajectories from videos of PD assessments. Features of the movement trajectories were used to detect and estimate the severity of parkinsonism and LID using random forest. Communication and drinking tasks were used to assess LID, while leg agility and toe tapping tasks were used to assess parkinsonism. Feature sets from tasks were also combined to predict total UDysRS and UPDRS Part III scores. Results: For LID, the communication task yielded the best results for dyskinesia (severity estimation: r = 0.661, detection: AUC = 0.930). For parkinsonism, leg agility had better results for severity estimation (r = 0.618), while toe tapping was better for detection (AUC = 0.773). UDysRS and UPDRS Part III scores were predicted with r = 0.741 and 0.530, respectively. Conclusion: This paper presents the first application of deep learning for vision-based assessment of parkinsonism and LID and demonstrates promising performance for the future translation of deep learning to PD clinical practices. Significance: The proposed system provides insight into the potential of computer vision and deep learning for clinical application in PD.Comment: 8 pages, 1 figure. Under revie