Real-World Repetition Estimation by Div, Grad and Curl

We consider the problem of estimating repetition in video, such as performing push-ups, cutting a melon or playing violin. Existing work shows good results under the assumption of static and stationary periodicity. As realistic video is rarely perfectly static and stationary, the often preferred Fourier-based measurements is inapt. Instead, we adopt the wavelet transform to better handle non-static and non-stationary video dynamics. From the flow field and its differentials, we derive three fundamental motion types and three motion continuities of intrinsic periodicity in 3D. On top of this, the 2D perception of 3D periodicity considers two extreme viewpoints. What follows are 18 fundamental cases of recurrent perception in 2D. In practice, to deal with the variety of repetitive appearance, our theory implies measuring time-varying flow and its differentials (gradient, divergence and curl) over segmented foreground motion. For experiments, we introduce the new QUVA Repetition dataset, reflecting reality by including non-static and non-stationary videos. On the task of counting repetitions in video, we obtain favorable results compared to a deep learning alternative

Decomposition and dictionary learning for 3D trajectories

International audienceA new model for describing a three-dimensional (3D) trajectory is proposed in this paper. The studied trajectory is viewed as a linear combination of rotatable 3D patterns. The resulting model is thus 3D rotation invariant (3DRI). Moreover, the temporal patterns are considered as shift-invariant. This paper is divided into two parts based on this model. On the one hand, the 3DRI decomposition estimates the active patterns, their coefficients, their rotations and their shift parameters. Based on sparse approximation, this is carried out by two non-convex optimizations: 3DRI matching pursuit (3DRI-MP) and 3DRI orthogonal matching pursuit (3DRI-OMP). On the other hand, a 3DRI learning method learns the characteristic patterns of a database through a 3DRI dictionary learning algorithm (3DRI-DLA). The proposed algorithms are first applied to simulation data to evaluate their performances and to compare them to other algorithms. Then, they are applied to real motion data of cued speech, to learn the 3D trajectory patterns characteristic of this gestural language

Characterizing flows with an instrumented particle measuring Lagrangian accelerations

We present in this article a novel Lagrangian measurement technique: an instrumented particle which continuously transmits the force/acceleration acting on it as it is advected in a flow. We develop signal processing methods to extract information on the flow from the acceleration signal transmitted by the particle. Notably, we are able to characterize the force acting on the particle and to identify the presence of a permanent large-scale vortex structure. Our technique provides a fast, robust and efficient tool to characterize flows, and it is particularly suited to obtain Lagrangian statistics along long trajectories or in cases where optical measurement techniques are not or hardly applicable.Comment: submitted to New Journal of Physic

Geometric Wavelet Scattering Networks on Compact Riemannian Manifolds

The Euclidean scattering transform was introduced nearly a decade ago to improve the mathematical understanding of convolutional neural networks. Inspired by recent interest in geometric deep learning, which aims to generalize convolutional neural networks to manifold and graph-structured domains, we define a geometric scattering transform on manifolds. Similar to the Euclidean scattering transform, the geometric scattering transform is based on a cascade of wavelet filters and pointwise nonlinearities. It is invariant to local isometries and stable to certain types of diffeomorphisms. Empirical results demonstrate its utility on several geometric learning tasks. Our results generalize the deformation stability and local translation invariance of Euclidean scattering, and demonstrate the importance of linking the used filter structures to the underlying geometry of the data.Comment: 35 pages; 3 figures; 2 tables; v3: Revisions based on reviewer comment