Real-World Repetition Estimation by Div, Grad and Curl

We consider the problem of estimating repetition in video, such as performing push-ups, cutting a melon or playing violin. Existing work shows good results under the assumption of static and stationary periodicity. As realistic video is rarely perfectly static and stationary, the often preferred Fourier-based measurements is inapt. Instead, we adopt the wavelet transform to better handle non-static and non-stationary video dynamics. From the flow field and its differentials, we derive three fundamental motion types and three motion continuities of intrinsic periodicity in 3D. On top of this, the 2D perception of 3D periodicity considers two extreme viewpoints. What follows are 18 fundamental cases of recurrent perception in 2D. In practice, to deal with the variety of repetitive appearance, our theory implies measuring time-varying flow and its differentials (gradient, divergence and curl) over segmented foreground motion. For experiments, we introduce the new QUVA Repetition dataset, reflecting reality by including non-static and non-stationary videos. On the task of counting repetitions in video, we obtain favorable results compared to a deep learning alternative

Context-aware and Scale-insensitive Temporal Repetition Counting

Temporal repetition counting aims to estimate the number of cycles of a given repetitive action. Existing deep learning methods assume repetitive actions are performed in a fixed time-scale, which is invalid for the complex repetitive actions in real life. In this paper, we tailor a context-aware and scale-insensitive framework, to tackle the challenges in repetition counting caused by the unknown and diverse cycle-lengths. Our approach combines two key insights: (1) Cycle lengths from different actions are unpredictable that require large-scale searching, but, once a coarse cycle length is determined, the variety between repetitions can be overcome by regression. (2) Determining the cycle length cannot only rely on a short fragment of video but a contextual understanding. The first point is implemented by a coarse-to-fine cycle refinement method. It avoids the heavy computation of exhaustively searching all the cycle lengths in the video, and, instead, it propagates the coarse prediction for further refinement in a hierarchical manner. We secondly propose a bidirectional cycle length estimation method for a context-aware prediction. It is a regression network that takes two consecutive coarse cycles as input, and predicts the locations of the previous and next repetitive cycles. To benefit the training and evaluation of temporal repetition counting area, we construct a new and largest benchmark, which contains 526 videos with diverse repetitive actions. Extensive experiments show that the proposed network trained on a single dataset outperforms state-of-the-art methods on several benchmarks, indicating that the proposed framework is general enough to capture repetition patterns across domains.Comment: Accepted by CVPR202

Human Gait Modeling, Prediction and Classification for Level Walking Using Harmonic Models Derived from a Single Thigh-Mounted IMU

The majority of human gait modeling is based on hip, foot or thigh acceleration. The regeneration accuracy of these modeling approaches is not very high. This paper presents a harmonic approach to modeling human gait during level walking based on gyroscopic signals for a single thigh-mounted Inertial Measurement Unit (IMU) and the flexion–extension derived from a single thigh-mounted IMU. The thigh angle can be modeled with five significant harmonics, with a regeneration accuracy of over 0.999 correlation and less than 0.5◦ RMSE per stride cycle. Comparable regeneration accuracies can be achieved with nine significant harmonics for the gyro signal. The fundamental frequency of the harmonic model can be estimated using the stride time, with an error level of 0.0479% (±0.0029%). Six commonly observed stride patterns, and harmonic models of thigh angle and gyro signal for those stride patterns, are presented in this paper. These harmonic models can be used to predict or classify the strides of walking trials, and the results are presented herein. Harmonic models may also be used for activity recognition. It has shown that human gait in level walking can be modeled with a harmonic model of thigh angle or gyro signal, using a single thigh-mounted IMU, to higher accuracies than existing techniques

Human gait modelling with step estimation and phase classification utilising a single thigh mounted IMU for vision impaired indoor navigation

This research is focused on human gait modelling for infrastructure free inertial navigation for vision impaired. A pedometer based on a single thigh mounted gyroscope, an efficient algorithm to estimate thigh flexion and extension, gait models for level walking, a model to estimate step length and a technique to detect gait phases based on a single thigh mounted Inertial Measurement Unit (IMU) were developed and confirmed higher accuracies