Deep Learning Based Abnormal Gait Classification System Study with Heterogeneous Sensor Network

Gait is one of the important biological characteristics of the human body. Abnormal gait is mostly related to the lesion site and has been demonstrated to play a guiding role in clinical research such as medical diagnosis and disease prevention. In order to promote the research of automatic gait pattern recognition, this paper introduces the research status of abnormal gait recognition and systems analysis of the common gait recognition technologies. Based on this, two gait information extraction methods, sensor-based and vision-based, are studied, including wearable system design and deep neural network-based algorithm design. In the sensor-based study, we proposed a lower limb data acquisition system. The experiment was designed to collect acceleration signals and sEMG signals under normal and pathological gaits. Specifically, wearable hardware-based on MSP430 and upper computer software based on Labview is designed. The hardware system consists of EMG foot ring, high-precision IMU and pressure-sensitive intelligent insole. Data of 15 healthy persons and 15 hemiplegic patients during walking were collected. The classification of gait was carried out based on sEMG and the average accuracy rate can reach 92.8% for CNN. For IMU signals five kinds of abnormal gait are trained based on three models: BPNN, LSTM, and CNN. The experimental results show that the system combined with the neural network can classify different pathological gaits well, and the average accuracy rate of the six-classifications task can reach 93%. In vision-based research, by using human keypoint detection technology, we obtain the precise location of the key points through the fusion of thermal mapping and offset, thus extracts the space-time information of the key points. However, the results show that even the state-of-the-art is not good enough for replacing IMU in gait analysis and classification. The good news is the rhythm wave can be observed within 2 m, which proves that the temporal and spatial information of the key points extracted is highly correlated with the acceleration information collected by IMU, which paved the way for the visual-based abnormal gait classification algorithm.步态指人走路时表现出来的姿态，是人体重要生物特征之一。异常步态多与病变部位有关，作为反映人体健康状况和行为能力的重要特征，其被论证在医疗诊断、疾病预防等临床研究中具有指导作用。为了促进步态模式自动识别的研究，本文介绍了异常步态识别的研究现状，系统地分析了常见步态识别技术以及算法，以此为基础研究了基于传感器与基于视觉两种步态信息提取方法，内容包括可穿戴系统设计与基于深度神经网络的算法设计。 在基于传感器的研究中，本工作开发了下肢步态信息采集系统，并利用该信息采集系统设计实验，采集正常与不同病理步态下的加速度信号与肌电信号，搭建深度神经网络完成分类任务。具体的，在系统搭建部分设计了基于MSP430的可穿戴硬件设备以及基于Labview的上位机软件，该硬件系统由肌电脚环，高精度IMU以及压感智能鞋垫组成，该上位机软件接收、解包蓝牙数据并计算出步频步长等常用步态参数。 在基于运动信号与基于表面肌电的研究中，采集了15名健康人与15名偏瘫病人的步态数据，并针对表面肌电信号训练卷积神经网络进行帕金森步态的识别与分类，平均准确率可达92.8%。针对运动信号训练了反向传播神经网络，LSTM以及卷积神经网络三种模型进行五种异常步态的分类任务。实验结果表明，本工作中步态信息采集系统结合神经网络模型，可以很好地对不同病理步态进行分类，六分类平均正确率可达93%。 在基于视觉的研究中，本文利用人体关键点检测技术，首先检测出图片中的一个或多个人，接着对边界框做图像分割，接着采用全卷积resnet对每一个边界框中的人物的主要关节点做热力图并分析偏移量，最后通过热力图与偏移的融合得到关键点的精确定位。通过该算法提取了不同步态下姿态关键点时空信息，为基于视觉的步态分析系统提供了基础条件。但实验结果表明目前最高准确率的人体关键点检测算法不足以替代IMU实现步态分析与分类。但在2m之内可以观察到节律信息，证明了所提取的关键点时空信息与IMU采集的加速度信息呈现较高相关度，为基于视觉的异常步态分类算法铺平了道路

Foot pressure distributions during walking in African elephants (Loxodonta africana)

Elephants, the largest living land mammals, have evolved a specialized foot morphology to help reduce locomotor pressures while supporting their large body mass. Peak pressures that could cause tissue damage are mitigated passively by the anatomy of elephants' feet, yet this mechanism does not seem to work well for some captive animals. This study tests how foot pressures vary among African and Asian elephants from habitats where natural substrates predominate but where foot care protocols differ. Variations in pressure patterns might be related to differences in husbandry, including but not limited to trimming and the substrates that elephants typically stand and move on. Both species' samples exhibited the highest concentration of peak pressures on the lateral digits of their feet (which tend to develop more disease in elephants) and lower pressures around the heel. The trajectories of the foot's centre of pressure were also similar, confirming that when walking at similar speeds, both species load their feet laterally at impact and then shift their weight medially throughout the step until toe-off. Overall, we found evidence of variations in foot pressure patterns that might be attributable to husbandry and other causes, deserving further examination using broader, more comparable samples

Master of Science

thesisAdvances in the field of robotics have laid a solid foundation for human-robot-interaction research; this research values demonstrations of emotional competence from robotic systems and herein lie opportunities for progress within the therapeutic industry, creation of companion robots, and integration of robotics among everyday households. The development of emotive expression within robotics is progressing at a fair pace; however, there is next to no research on this form of expression as it pertains to a robot's manner of walking. The work presented here proves that it is possible for robots to walk with the capability of expressing emotions that are identifiable by their human counterparts. This hypothesis is explored utilizing a four-legged robot in simulation and reality, and the details necessary for this application are presented in this work. This quadruped is comprised of four manipulators each consisting of seven degrees of freedom. The inverse kinematics and dynamics are solved for each leg with closed form solutions that incorporate the inverse of Euler's finite rotation formula. With the kinematics solved, the robot utilizes a central pattern generator to create a neutral gait and balances with an augmented center of pressure that closely resembles the zero moment point algorithm. Independent of the kinematics, a method of generating poses that represent the emotions: happy, sad, angry, and fearful, is presented. This work also details how to overlay poses atop a gait to transform the neutral gait into an emotive walking style. In addition to laying the framework for developing the emotive walking styles, an evaluation of the presented gaits is detailed. Two IRB approved studies were performed independently of each other. The first study took feedback from subjects regarding ways to make the emotive gaits more compelling and applied them to the initial poses. The second study evaluated the effectiveness of the final gaits, with improved poses, and proves that emotive walking patterns were created; walking patterns that will be suitable for emotional acuity

Speed-independent gait identification for mobile devices

Due to the intensive use of mobile phones for diferent purposes, these devices usually contain condential information which must not be accessed by another person apart from the owner of the device. Furthermore, the new generation phones commonly incorporate an accelerometer which may be used to capture the acceleration signals produced as a result of owner s gait. Nowadays, gait identication in basis of acceleration signals is being considered as a new biometric technique which allows blocking the device when another person is carrying it. Although distance based approaches as Euclidean distance or dynamic time warping have been applied to solve this identication problem, they show di±culties when dealing with gaits at diferent speeds. For this reason, in this paper, a method to extract an average template from instances of the gait at diferent velocities is presented. This method has been tested with the gait signals of 34 subjects while walking at diferent motion speeds (slow, normal and fast) and it has shown to improve the performance of Euclidean distance and classical dynamic time warping

Non-Intrusive Gait Recognition Employing Ultra Wideband Signal Detection

A self-regulating and non-contact impulse radio ultra wideband (IR-UWB) based 3D human gait analysis prototype has been modeled and developed with the help of supervised machine learning (SML) for this application for the first time. The work intends to provide a rewarding assistive biomedical application which would help doctors and clinicians monitor human gait trait and abnormalities with less human intervention in the fields of physiological examinations, physiotherapy, home assistance, rehabilitation success determination and health diagnostics, etc. The research comprises IR-UWB data gathered from a number of male and female participants in both anechoic chamber and multi-path environments. In total twenty four individuals have been recruited, where twenty individuals were said to have normal gait and four persons complained of knee pain that resulted in compensated spastic walking patterns. A 3D postural model of human movements has been created from the backscattering property of the radar pulses employing understanding of spherical trigonometry and vector fields. This subjective data (height of the body areas from the ground) of an individual have been recorded and implemented to extract the gait trait from associated biomechanical activity and differentiates the lower limb movement patterns from other body areas. Initially, a 2D postural model of human gait is presented from IR-UWB sensing phenomena employing spherical co-ordinate and trigonometry where only two dimensions such as, distance from radar and height of reflection have been determined. There are five pivotal gait parameters; step frequency, cadence, step length, walking speed, total covered distance, and body orientation which have all been measured employing radar principles and short term Fourier transformation (STFT). Subsequently, the proposed gait identification and parameter characterization has been analysed, tested and validated against popularly accepted smartphone applications with resulting variations of less than 5%. Subsequently, the spherical trigonometric model has been elevated to a 3D postural model where the prototype can determine width of motion, distance from radar, and height of reflection. Vector algebra has been incorporated with this 3D model to measure knee angles and hip angles from the extension and flexion of lower limbs to understand the gait behavior throughout the entire range of bipedal locomotion. Simultaneously, the Microsoft Kinect Xbox One has been employed during the experiment to assist in the validation process. The same vector mathematics have been implemented to the skeleton data obtained from Kinect to determine both the hip and knee angles. The outcomes have been compared by statistical graphical approach Bland and Altman (B&A) analysis. Further, the changes of knee angles obtained from the normal gaits have been used to train popular SMLs such as, k-nearest neighbour (kNN) and support vector machines (SVM). The trained model has subsequently been tested with the new data (knee angles extracted from both normal and abnormal gait) to assess the prediction ability of gait abnormality recognition. The outcomes have been validated through standard and wellknown statistical performance metrics with promising results found. The outcomes prove the acceptability of the proposed non-contact IR-UWB gait recognition to detect gait

Principal component analysis for human gait recognition system

This paper represents a method for Human Recognition system using Principal Component Analysis. Human Gait recognition works on the gait of walking subjects to identify people without them knowing or without their permission. The initial step in this kind of system is to generate silhouette frames of walking human. A number of features couldb be exytacted from these frames such as centriod ratio, heifht, width and orientation. The Principal Component Analysis (PCA) is used for the extracted features to condense the information and produces the main components that can represent the gait sequences for each waiking human. In the testing phase, the generated gait sequences are recognized by using a minimum distance classifier based on eluclidean distance matched with the one that already exist in the database used to identify walking subject

Evaluation of Pedometer Performance Across Multiple Gait Types Using Video for Ground Truth

This dissertation is motivated by improving healthcare through the development of wearable sensors. This work seeks improvement in the evaluation and development of pedometer algorithms, and is composed of two chapters describing the collection of the dataset and describing the im-plementation and evaluation of three previously developed pedometer algorithms on the dataset collected. Our goal is to analyze pedometer algorithms under more natural conditions that occur during daily living where gaits are frequently changing or remain regular for only brief periods of time. We video recorded 30 participants performing 3 activities: walking around a track, walking through a building, and moving around a room. The ground truth time of each step was manu-ally marked in the accelerometer signals through video observation. Collectively 60,853 steps were recorded and annotated. A subclass of steps called shifts were identified as those occurring at the beginning and end of regular strides, during gait changes, and during pivots changing the direction of motion. While shifts comprised only .03% of steps in the regular stride activity, they comprised 10-25% of steps in the semi-regular and unstructured activities. We believe these motions should be identified separately, as they provide different accelerometer signals, and likely result in different amounts of energy expenditure. This dataset will be the first to specifically allow for pedometer algorithms to be evaluated on unstructured gaits that more closely model natural activities. In order to provide pilot evaluation data, a commercial pedometer, the Fitbit Charge 2, and three prior step detection algorithms were analyzed. The Fitbit consistently underestimated the total number of steps taken across each gait type. Because the Fitbit algorithm is proprietary, it could not be reimplemented and examined beyond a raw step count comparison. Three previously published step detection algorithms, however, were implemented and examined in detail on the dataset. The three algorithms are based on three different methods of step detection; peak detection, zero crossing (threshold based), and autocorrelation. The evaluation of these algorithms was performed across 5 dimensions, including algorithm, parameter set, gait type, sensor position, and evaluation metric, which yielded 108 individual measures of accuracy. Accuracy across each of the 5 dimensions were examined individually in order to determine trends. In general, training parameters to this dataset caused a significant accuracy improvement. The most accurate algorithm was dependent on gait type, sensor position, and evaluation metric, indicating no clear “best approach” to step detection. In general, algorithms were most accurate for regular gait and least accurate for unstructured gait. In general, accuracy was higher for hip and ankle worn sensors than it was for wrist worn sensors. Finally, evaluation across running count accuracy (RCA) and step detection accuracy (SDA) revealed similar trends across gait type and sensor position, but each metric indicated a different algorithm with the highest overall accuracy. A classifier was developed to identify gait type in an effort to use this information to improve pedometer accuracy. The classifier’s features are based on the Fast Fourier Transform (FFT) applied to the accelerometer data gathered from each sensor throughout each activity. A peak detector was developed to identify the maximum value of the FFT, the width of the peak yielding the maximum value, and the number of peaks in each FFT. These features were then applied to a Naive Bayes classifier, which correctly identified the gait (regular, semi-regular, or unstructured) with 84% accuracy. A varying algorithm pedometer was then developed which switched between the peak detection, threshold crossing, and autocorrelation based algorithms depending on which algorithm performed best for the sensor location and detected gait type. This process yielded a step detection accuracy of 84%. This was a 3% improvement when compared to the greatest accuracy achieved by the best performing algorithm, the peak detection algorithm. It was also identified that in order to provide quicker real-time transitions between algorithms, the data should be examined in smaller windows. Window sizes of 3, 5, 8, 10, 15, 20, and 30 seconds were tested, and the highest overall accuracy was found for a window size of 5 seconds. These smaller windows of time included behaviors which do not correspond directly with the regular, semi-regular, and unstructured gait activities. Instead, three stride types were identified: steady stride, irregular stride, and idle. These stride types were identified with 82% accuracy. This experiment showed that at an activity level, gait detection can improve pedometer accuracy and indicated that applying the same principles to a smaller window size could allow for more responsive real-time algorithm selection