Distance‐based time series classification approach for task recognition with application in surgical robot autonomy

BackgroundRobotic‐assisted surgery allows surgeons to perform many types of complex operations with greater precision than is possible with conventional surgery. Despite these advantages, in current systems, a surgeon should communicate with the device directly and manually. To allow the robot to adjust parameters such as camera position, the system needs to know automatically what task the surgeon is performing.MethodsA distance‐based time series classification framework has been developed which measures dynamic time warping distance between temporal trajectory data of robot arms and classifies surgical tasks and gestures using a k‐nearest neighbor algorithm.ResultsResults on real robotic surgery data show that the proposed framework outperformed state‐of‐the‐art methods by up to 9% across three tasks and by 8% across gestures.ConclusionThe proposed framework is robust and accurate. Therefore, it can be used to develop adaptive control systems that will be more responsive to surgeons’ needs by identifying next movements of the surgeon. Copyright © 2016 John Wiley & Sons, Ltd.Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/138333/1/rcs1766.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/138333/2/rcs1766_am.pd

Quantitative Estimation of Movement Progress during Rehabilitation after Knee/Hip Replacement Surgery

Mobility improvement for patients is one of the primary concerns of physiotherapy rehabilitation. In a typical physiotherapy session, the patient is instructed to perform multiple exercises, based on a specific regimen recommended by the physiotherapist for each patient. The physiotherapist then evaluates the patient's progress based on his or her performance during the exercises. Providing the physiotherapist and the patient with a quantified and objective measure of progress, based on both individual exercises and the exercise set, can be beneficial for monitoring the patient's performance. The quantified measure can also be beneficial when the physiotherapist is not available, e.g., crowded gym or rehabilitation at home. In this thesis, two approaches are introduced for quantifying patient performance. One approach describes the movement timeseries by statistical measures and the other by a stochastic model. Both approaches formulate a distance between patient data and the healthy population as the measure of performance. Distance measures are defined to capture the performance of one repetition of an exercise or multiple repetitions of the same exercise. To capture patient progress across multiple exercises, a quality measure and overall score are formulated based on the distance measures and are used to quantify the overall performance for each session. The proposed approaches are compared to several existing approaches, including sample distribution approaches (two sample kernel), classifier-based approaches (Naive Bayes, Support Vector Machines, and Kullback-Leibler Divergence), and dynamical movement primitives. In their original formulation, existing approaches are not capable of estimating measures of performance for multiple exercises. Therefore, the measures of performance for multiple repetitions of the same exercise are estimated using the existing approaches, while the formulation proposed in this thesis is used to estimate the overall performance for multiple exercises in one session. The effects of different variabilities in human motion on the performance of the proposed approaches and the comparison approaches are investigated with both synthetic and patient data. The patient data consists of rehabilitation data recorded from patients recovering from knee or hip replacement surgery, the associated exercise regimen and physiotherapist evaluations of progress. The methods are evaluated quantitatively based on correlation between methods, correlation with exercise regimen difficulty, and qualitatively based on the patients' medical charts. The proposed approaches are capable of capturing the trend of progress for the synthetic dataset and are superior to the existing approaches in presence of multiple sources of variability. For patient data, the proposed approaches correlate moderately with the score obtained from the exercise regimen, and qualitatively correspond with the patients' medical charts. The results indicate that the quantified measures of progress obtained from the proposed approaches are promising tools for supporting physiotherapy practice through monitoring patient progress.1 yea

Assistive Technology and Biomechatronics Engineering

This Special Issue will focus on assistive technology (AT) to address biomechanical and control of movement issues in individuals with impaired health, whether as a result of disability, disease, or injury. All over the world, technologies are developed that make human life richer and more comfortable. However, there are people who are not able to benefit from these technologies. Research can include development of new assistive technology to promote more effective movement, the use of existing technology to assess and treat movement disorders, the use and effectiveness of virtual rehabilitation, or theoretical issues, such as modeling, which underlie the biomechanics or motor control of movement disorders. This Special Issue will also cover Internet of Things (IoT) sensing technology and nursing care robot applications that can be applied to new assistive technologies. IoT includes data, more specifically gathering them efficiently and using them to enable intelligence, control, and new applications

A Semi-Supervised Approach for Kernel-Based Temporal Clustering

Temporal clustering refers to the partitioning of a time series into multiple non-overlapping segments that belong to k temporal clusters, in such a way that segments in the same cluster are more similar to each other than to those in other clusters. Temporal clustering is a fundamental task in many fields, such as computer animation, computer vision, health care, and robotics. The applications of temporal clustering in those areas are diverse, and include human-motion imitation and recognition, emotion analysis, human activity segmentation, automated rehabilitation exercise analysis, and human-computer interaction. However, temporal clustering using a completely unsupervised method may not produce satisfactory results. Similar to regular clustering, temporal clustering also benefits from some expert knowledge that may be available. The type of approach that utilizes a small amount of knowledge to “guide” the clustering process is known as “semi-supervised clustering.” Semi-supervised temporal clustering is a strategy in which extra knowledge, in the form of pairwise constraints, is incorporated into the temporal data to help with the partitioning problem. This thesis proposes a process to adapt and transform two kernel-based methods into semi-supervised temporal clustering methods. The proposed process is exclusive to kernel-based clustering methods, and is based on two concepts. First, it uses the idea of instance-level constraints, in the form of must-link and cannot-link, to supervise the clustering methods. Second, it uses a dynamic-programming method to search for the optimal temporal clusters. The proposed process is applied to two algorithms, aligned cluster analysis (ACA) and spectral clustering. To validate the advantages of the proposed temporal semi-supervised clustering methods, a comparative analysis was performed, using the original versions of the algorithm and another semi-supervised temporal cluster. This evaluation was conducted with both synthetic data and two real-world applications. The first application includes two naturalistic audio-visual human emotion datasets, and the second application focuses on human-motion segmentation. Results show substantial improvements in accuracy, with minimal supervision, compared to unsupervised and other temporal semi-supervised approaches, without compromising time performance

Segmenting Human Motion for Automated Rehabilitation Exercise Analysis

Abstract — This paper proposes an approach for the automated segmentation and identification of movement segments from continuous time series data of human movement, collected through motion capture of ambulatory sensors. The proposed approach uses a two stage identification and recognition process, based on velocity and stochastic modeling of each motion to be identified. In the first stage, motion segment candidates are identified based on a unique sequence of velocity features such as velocity peaks and zero velocity crossings. In the second stage, Hidden Markov models are used to accurately identify segment locations from the identified candidates. The approach is capable of on-line segmentation and identification, enabling interactive feedback in rehabilitation applications. The approach is validated on a rehabilitation movement dataset, and achieves a segmentation accuracy of 89%. I