Remote Gait type classification system using markerless 2D video

Several pathologies can alter the way people walk, i.e., their gait. Gait analysis can be used to detect such alterations and, therefore, help diagnose certain pathologies or assess people’s health and recovery. Simple vision-based systems have a considerable potential in this area, as they allow the capture of gait in unconstrained environments, such as at home or in a clinic, while the required computations can be done remotely. State-of-the-art vision-based systems for gait analysis use deep learning strategies, thus requiring a large amount of data for training. However, to the best of our knowledge, the largest publicly available pathological gait dataset contains only 10 subjects, simulating 5 types of gait. This paper presents a new dataset, GAIT-IT, captured from 21 subjects simulating 5 types of gait, at 2 severity levels. The dataset is recorded in a professional studio, making the sequences free of background camouflage, variations in illumination and other visual artifacts. The dataset is used to train a novel automatic gait analysis system. Compared to the state-of-the-art, the proposed system achieves a drastic reduction in the number of trainable parameters, memory requirements and execution times, while the classification accuracy is on par with the state-of-the-art. Recognizing the importance of remote healthcare, the proposed automatic gait analysis system is integrated with a prototype web application. This prototype is presently hosted in a private network, and after further tests and development it will allow people to upload a video of them walking and execute a web service that classifies their gait. The web application has a user-friendly interface usable by healthcare professionals or by laypersons. The application also makes an association between the identified type of gait and potential gait pathologies that exhibit the identified characteristics.info:eu-repo/semantics/publishedVersio

Knee joint sagittal plane movement in cerebral palsy: a comparative study of 2-dimensional markerless video and 3-dimensional gait analysis

8Background and purpose — Gait analysis is indicated in children with cerebral palsy (CP) to identify and quantify gait deviations. One particularly difficult-to-treat deviation, crouch gait, can progress in adolescence and ultimately limit the ability to ambulate. An objective quantitative assessment is essential to early identify progressive gait impairments in children with CP. 3-dimensional gait analysis (3D GA) is considered the gold standard, although it is expensive, seldom available, and unnecessarily detailed for screening and follow-up. Simple video assessments are time-consuming when processed manually, but more convenient if used in conjunction with video processing algorithms; this has yet been validated in CP. We validate a 2-dimensional markerless (2D ML) assessment of knee joint flexion/extension angles of the gait cycle in children and young adults with CP. Patients and methods — 18 individuals, mean age 15 years (6.5–28), participated. 11 had bilateral, 3 unilateral, 3 dyskinetic, and 1 ataxic CP. In the Gross Motor Function Classification System, 6 were at level I, 11 at level II, and 1 at level III. We compared 2D ML, using a single video camera with computer processing, and 3D GA. Results — The 2D ML method overestimated the knee flexion/extension angle values by 3.3 to 7.0 degrees compared with 3D GA. The reliability within 2D ML and 3D GA was mostly good to excellent. Interpretation — Despite overestimating, 2D ML is a reliable and convenient tool to assess knee angles and, more importantly, to detect changes over time within a follow-up program in ambulatory children with CP.openopenPantzar-Castilla, Evelina; Cereatti, Andrea; Figari, Giulio; Valeri, Nicolò; Paolini, Gabriele; Della Croce, Ugo; Magnuson, Anders; Riad, JacquesPantzar-Castilla, Evelina; Cereatti, Andrea; Figari, Giulio; Valeri, Nicolò; Paolini, Gabriele; Della Croce, Ugo; Magnuson, Anders; Riad, Jacque

A spatiotemporal deep learning approach for automatic pathological Gait classification

Human motion analysis provides useful information for the diagnosis and recovery assessment of people suffering from pathologies, such as those affecting the way of walking, i.e., gait. With recent developments in deep learning, state-of-the-art performance can now be achieved using a single 2D-RGB-camera-based gait analysis system, offering an objective assessment of gait-related pathologies. Such systems provide a valuable complement/alternative to the current standard practice of subjective assessment. Most 2D-RGB-camera-based gait analysis approaches rely on compact gait representations, such as the gait energy image, which summarize the characteristics of a walking sequence into one single image. However, such compact representations do not fully capture the temporal information and dependencies between successive gait movements. This limitation is addressed by proposing a spatiotemporal deep learning approach that uses a selection of key frames to represent a gait cycle. Convolutional and recurrent deep neural networks were combined, processing each gait cycle as a collection of silhouette key frames, allowing the system to learn temporal patterns among the spatial features extracted at individual time instants. Trained with gait sequences from the GAIT-IT dataset, the proposed system is able to improve gait pathology classification accuracy, outperforming state-of-the-art solutions and achieving improved generalization on cross-dataset tests.info:eu-repo/semantics/publishedVersio

Markerless Human Motion Analysis

Measuring and understanding human motion is crucial in several domains, ranging from neuroscience, to rehabilitation and sports biomechanics. Quantitative information about human motion is fundamental to study how our Central Nervous System controls and organizes movements to functionally evaluate motor performance and deficits. In the last decades, the research in this field has made considerable progress. State-of-the-art technologies that provide useful and accurate quantitative measures rely on marker-based systems. Unfortunately, markers are intrusive and their number and location must be determined a priori. Also, marker-based systems require expensive laboratory settings with several infrared cameras. This could modify the naturalness of a subject\u2019s movements and induce discomfort. Last, but not less important, they are computationally expensive in time and space. Recent advances on markerless pose estimation based on computer vision and deep neural networks are opening the possibility of adopting efficient video-based methods for extracting movement information from RGB video data. In this contest, this thesis presents original contributions to the following objectives: (i) the implementation of a video-based markerless pipeline to quantitatively characterize human motion; (ii) the assessment of its accuracy if compared with a gold standard marker-based system; (iii) the application of the pipeline to different domains in order to verify its versatility, with a special focus on the characterization of the motion of preterm infants and on gait analysis. With the proposed approach we highlight that, starting only from RGB videos and leveraging computer vision and machine learning techniques, it is possible to extract reliable information characterizing human motion comparable to that obtained with gold standard marker-based systems

Testing the performance of an innovative markerless technique for quantitative and qualitative gait analysis

Gait abnormalities such as high stride and step frequency/cadence (SF-stride/second, CAD-step/second), stride variability (SV) and low harmony may increase the risk of injuries and be a sentinel of medical conditions. This research aims to present a new markerless video-based technology for quantitative and qualitative gait analysis. 86 healthy individuals (mead age 32 years) performed a 90 s test on treadmill at self-selected walking speed. We measured SF and CAD by a photoelectric sensors system; then, we calculated average \ub1 standard deviation (SD) and within-subject coefficient of variation (CV) of SF as an index of SV. We also recorded a 60 fps video of the patient. With a custom-designed web-based video analysis software, we performed a spectral analysis of the brightness over time for each pixel of the image, that reinstituted the frequency contents of the videos. The two main frequency contents (F1 and F2) from this analysis should reflect the forcing/dominant variables, i.e., SF and CAD. Then, a harmony index (HI) was calculated, that should reflect the proportion of the pixels of the image that move consistently with F1 or its supraharmonics. The higher the HI value, the less variable the gait. The correspondence SF-F1 and CAD-F2 was evaluated with both paired t-Test and correlation and the relationship between SV and HI with correlation. SF and CAD were not significantly different from and highly correlated with F1 (0.893 \ub1 0.080 Hz vs. 0.895 \ub1 0.084 Hz, p < 0.001, r2 = 0.99) and F2 (1.787 \ub1 0.163 Hz vs. 1.791 \ub1 0.165 Hz, p < 0.001, r2 = 0.97). The SV was 1.84% \ub1 0.66% and it was significantly and moderately correlated with HI (0.082 \ub1 0.028, p < 0.001, r2 = 0.13). The innovative video-based technique of global, markerless gait analysis proposed in our study accurately identifies the main frequency contents and the variability of gait in healthy individuals, thus providing a time-efficient, low-cost means to quantitatively and qualitatively study human locomotion

Using transfer learning for classification of gait pathologies

Different diseases can affect an individual’s gait in different ways and, therefore, gait analysis can provide important insights into an individual’s health and well-being. Currently, most systems that perform gait analysis using 2D video are limited to simple binary classification of gait as being either normal or impaired. While some systems do perform gait classification across different pathologies, the reported results still have a considerable margin for improvement. This paper presents a novel system that performs classification of gait across different pathologies, with considerably improved results. The system computes the walking individual’s silhouettes, which are computed from a 2D video sequence, and combines them into a representation known as the gait energy image (GEI), which provides robustness against silhouette segmentation errors. In this work, instead of using a set of handcrafted gait features, feature extraction is done using the VGG-19 convolutional neural network. The network is fine-tuned to automatically extract the features that best represent gait pathologies, using transfer learning. The use of transfer learning improves the classification accuracy while avoiding the need of a very large training set, as the network is pre-trained for generic image description, which also contributes to a better generalization when tested across different datasets. The proposed system performs the final classification using linear discriminant analysis (LDA). Obtained results show that the proposed system outperforms the state-of-the-art, achieving a classification accuracy of 95% on a dataset containing gait sequences affected by diplegia, hemiplegia, neuropathy and Parkinson’s disease, along with normal gait sequences.info:eu-repo/semantics/acceptedVersio

Development of Markerless Systems for Automatic Analysis of Movements and Facial Expressions: Applications in Neurophysiology

This project is focused on the development of markerless methods for studying facial expressions and movements in neurology, focusing on Parkinson’s disease (PD) and disorders of consciousness (DOC). PD is a neurodegenerative illness that affects around 2% of the population over 65 years old. Impairments of voice/speech are among the main signs of PD. This set of impairments is called hypokinetic dysarthria, because of the reduced range of movements involved in speech. This reduction can be visible also in other facial muscles, leading to a hypomimia. Despite the high percentage of patients that suffer from dysarthria and hypomimia, only a few of them undergo speech therapy with the aim to improve the dynamic of articulatory/facial movements. The main reason is the lack of low cost methodologies that could be implemented at home. DOC after coma are Vegetative State (VS), characterized by the absence of self-awareness and awareness of the environment, and Minimally Conscious State (MCS), in which certain behaviors are sufficiently reproducible to be distinguished from reflex responses. The differential diagnosis between VS and MCS can be hard and prone to a high rate of misdiagnosis (~40%). This differential diagnosis is mainly based on neuro-behavioral scales. A key role to plan the rehabilitation in DOC patients is played by the first diagnosis after coma. In fact, MCS patients are more prone to a consciousness recovery than VS patients. Concerning PD the aim is the development of contactless systems that could be used to study symptoms related to speech and facial movements/expressions. The methods proposed here, based on acoustical analysis and video processing techniques could support patients during speech therapy also at home. Concerning DOC patients the project is focused on the assessment of reflex and cognitive responses to standardized stimuli. This would allow objectifying the perceptual analysis performed by clinicians

Deep learning-based behavioral profiling of rodent stroke recovery

BACKGROUND Stroke research heavily relies on rodent behavior when assessing underlying disease mechanisms and treatment efficacy. Although functional motor recovery is considered the primary targeted outcome, tests in rodents are still poorly reproducible and often unsuitable for unraveling the complex behavior after injury. RESULTS Here, we provide a comprehensive 3D gait analysis of mice after focal cerebral ischemia based on the new deep learning-based software (DeepLabCut, DLC) that only requires basic behavioral equipment. We demonstrate a high precision 3D tracking of 10 body parts (including all relevant joints and reference landmarks) in several mouse strains. Building on this rigor motion tracking, a comprehensive post-analysis (with >100 parameters) unveils biologically relevant differences in locomotor profiles after a stroke over a time course of 3 weeks. We further refine the widely used ladder rung test using deep learning and compare its performance to human annotators. The generated DLC-assisted tests were then benchmarked to five widely used conventional behavioral set-ups (neurological scoring, rotarod, ladder rung walk, cylinder test, and single-pellet grasping) regarding sensitivity, accuracy, time use, and costs. CONCLUSIONS We conclude that deep learning-based motion tracking with comprehensive post-analysis provides accurate and sensitive data to describe the complex recovery of rodents following a stroke. The experimental set-up and analysis can also benefit a range of other neurological injuries that affect locomotion