Affine differential geometry analysis of human arm movements

Humans interact with their environment through sensory information and motor actions. These interactions may be understood via the underlying geometry of both perception and action. While the motor space is typically considered by default to be Euclidean, persistent behavioral observations point to a different underlying geometric structure. These observed regularities include the “two-thirds power law” which connects path curvature with velocity, and “local isochrony” which prescribes the relation between movement time and its extent. Starting with these empirical observations, we have developed a mathematical framework based on differential geometry, Lie group theory and Cartan’s moving frame method for the analysis of human hand trajectories. We also use this method to identify possible motion primitives, i.e., elementary building blocks from which more complicated movements are constructed. We show that a natural geometric description of continuous repetitive hand trajectories is not Euclidean but equi-affine. Specifically, equi-affine velocity is piecewise constant along movement segments, and movement execution time for a given segment is proportional to its equi-affine arc-length. Using this mathematical framework, we then analyze experimentally recorded drawing movements. To examine movement segmentation and classification, the two fundamental equi-affine differential invariants—equi-affine arc-length and curvature are calculated for the recorded movements. We also discuss the possible role of conic sections, i.e., curves with constant equi-affine curvature, as motor primitives and focus in more detail on parabolas, the equi-affine geodesics. Finally, we explore possible schemes for the internal neural coding of motor commands by showing that the equi-affine framework is compatible with the common model of population coding of the hand velocity vector when combined with a simple assumption on its dynamics. We then discuss several alternative explanations for the role that the equi-affine metric may play in internal representations of motion perception and production

Geometry-aware Manipulability Learning, Tracking and Transfer

Body posture influences human and robots performance in manipulation tasks, as appropriate poses facilitate motion or force exertion along different axes. In robotics, manipulability ellipsoids arise as a powerful descriptor to analyze, control and design the robot dexterity as a function of the articulatory joint configuration. This descriptor can be designed according to different task requirements, such as tracking a desired position or apply a specific force. In this context, this paper presents a novel \emph{manipulability transfer} framework, a method that allows robots to learn and reproduce manipulability ellipsoids from expert demonstrations. The proposed learning scheme is built on a tensor-based formulation of a Gaussian mixture model that takes into account that manipulability ellipsoids lie on the manifold of symmetric positive definite matrices. Learning is coupled with a geometry-aware tracking controller allowing robots to follow a desired profile of manipulability ellipsoids. Extensive evaluations in simulation with redundant manipulators, a robotic hand and humanoids agents, as well as an experiment with two real dual-arm systems validate the feasibility of the approach.Comment: Accepted for publication in the Intl. Journal of Robotics Research (IJRR). Website: https://sites.google.com/view/manipulability. Code: https://github.com/NoemieJaquier/Manipulability. 24 pages, 20 figures, 3 tables, 4 appendice

A Compact Representation of Drawing Movements with Sequences of Parabolic Primitives

Some studies suggest that complex arm movements in humans and monkeys may optimize several objective functions, while others claim that arm movements satisfy geometric constraints and are composed of elementary components. However, the ability to unify different constraints has remained an open question. The criterion for a maximally smooth (minimizing jerk) motion is satisfied for parabolic trajectories having constant equi-affine speed, which thus comply with the geometric constraint known as the two-thirds power law. Here we empirically test the hypothesis that parabolic segments provide a compact representation of spontaneous drawing movements. Monkey scribblings performed during a period of practice were recorded. Practiced hand paths could be approximated well by relatively long parabolic segments. Following practice, the orientations and spatial locations of the fitted parabolic segments could be drawn from only 2–4 clusters, and there was less discrepancy between the fitted parabolic segments and the executed paths. This enabled us to show that well-practiced spontaneous scribbling movements can be represented as sequences (“words”) of a small number of elementary parabolic primitives (“letters”). A movement primitive can be defined as a movement entity that cannot be intentionally stopped before its completion. We found that in a well-trained monkey a movement was usually decelerated after receiving a reward, but it stopped only after the completion of a sequence composed of several parabolic segments. Piece-wise parabolic segments can be generated by applying affine geometric transformations to a single parabolic template. Thus, complex movements might be constructed by applying sequences of suitable geometric transformations to a few templates. Our findings therefore suggest that the motor system aims at achieving more parsimonious internal representations through practice, that parabolas serve as geometric primitives and that non-Euclidean variables are employed in internal movement representations (due to the special role of parabolas in equi-affine geometry)

Quantifying the Evolutionary Self Structuring of Embodied Cognitive Networks

We outline a possible theoretical framework for the quantitative modeling of networked embodied cognitive systems. We notice that: 1) information self structuring through sensory-motor coordination does not deterministically occur in Rn vector space, a generic multivariable space, but in SE(3), the group structure of the possible motions of a body in space; 2) it happens in a stochastic open ended environment. These observations may simplify, at the price of a certain abstraction, the modeling and the design of self organization processes based on the maximization of some informational measures, such as mutual information. Furthermore, by providing closed form or computationally lighter algorithms, it may significantly reduce the computational burden of their implementation. We propose a modeling framework which aims to give new tools for the design of networks of new artificial self organizing, embodied and intelligent agents and the reverse engineering of natural ones. At this point, it represents much a theoretical conjecture and it has still to be experimentally verified whether this model will be useful in practice.

A Riemannian Take on Human Motion Analysis and Retargeting

Dynamic motions of humans and robots are widely driven by posture-dependent nonlinear interactions between their degrees of freedom. However, these dynamical effects remain mostly overlooked when studying the mechanisms of human movement generation. Inspired by recent works, we hypothesize that human motions are planned as sequences of geodesic synergies, and thus correspond to coordinated joint movements achieved with piecewise minimum energy. The underlying computational model is built on Riemannian geometry to account for the inertial characteristics of the body. Through the analysis of various human arm motions, we find that our model segments motions into geodesic synergies, and successfully predicts observed arm postures, hand trajectories, as well as their respective velocity profiles. Moreover, we show that our analysis can further be exploited to transfer arm motions to robots by reproducing individual human synergies as geodesic paths in the robot configuration space.Comment: Accepted for publication in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 202

Movement Timing and Invariance Arise from Several Geometries

Human movements show several prominent features; movement duration is nearly independent of movement size (the isochrony principle), instantaneous speed depends on movement curvature (captured by the 2/3 power law), and complex movements are composed of simpler elements (movement compositionality). No existing theory can successfully account for all of these features, and the nature of the underlying motion primitives is still unknown. Also unknown is how the brain selects movement duration. Here we present a new theory of movement timing based on geometrical invariance. We propose that movement duration and compositionality arise from cooperation among Euclidian, equi-affine and full affine geometries. Each geometry posses a canonical measure of distance along curves, an invariant arc-length parameter. We suggest that for continuous movements, the actual movement duration reflects a particular tensorial mixture of these canonical parameters. Near geometrical singularities, specific combinations are selected to compensate for time expansion or compression in individual parameters. The theory was mathematically formulated using Cartan's moving frame method. Its predictions were tested on three data sets: drawings of elliptical curves, locomotion and drawing trajectories of complex figural forms (cloverleaves, lemniscates and limaçons, with varying ratios between the sizes of the large versus the small loops). Our theory accounted well for the kinematic and temporal features of these movements, in most cases better than the constrained Minimum Jerk model, even when taking into account the number of estimated free parameters. During both drawing and locomotion equi-affine geometry was the most dominant geometry, with affine geometry second most important during drawing; Euclidian geometry was second most important during locomotion. We further discuss the implications of this theory: the origin of the dominance of equi-affine geometry, the possibility that the brain uses different mixtures of these geometries to encode movement duration and speed, and the ontogeny of such representations

Discovery and recognition of motion primitives in human activities

We present a novel framework for the automatic discovery and recognition of motion primitives in videos of human activities. Given the 3D pose of a human in a video, human motion primitives are discovered by optimizing the `motion flux', a quantity which captures the motion variation of a group of skeletal joints. A normalization of the primitives is proposed in order to make them invariant with respect to a subject anatomical variations and data sampling rate. The discovered primitives are unknown and unlabeled and are unsupervisedly collected into classes via a hierarchical non-parametric Bayes mixture model. Once classes are determined and labeled they are further analyzed for establishing models for recognizing discovered primitives. Each primitive model is defined by a set of learned parameters. Given new video data and given the estimated pose of the subject appearing on the video, the motion is segmented into primitives, which are recognized with a probability given according to the parameters of the learned models. Using our framework we build a publicly available dataset of human motion primitives, using sequences taken from well-known motion capture datasets. We expect that our framework, by providing an objective way for discovering and categorizing human motion, will be a useful tool in numerous research fields including video analysis, human inspired motion generation, learning by demonstration, intuitive human-robot interaction, and human behavior analysis