3-D Human Action Recognition by Shape Analysis of Motion Trajectories on Riemannian Manifold

International audienceRecognizing human actions in 3D video sequences is an important open problem that is currently at the heart of many research domains including surveillance, natural interfaces and rehabilitation. However, the design and development of models for action recognition that are both accurate and efficient is a challenging task due to the variability of the human pose, clothing and appearance. In this paper, we propose a new framework to extract a compact representation of a human action captured through a depth sensor, and enable accurate action recognition. The proposed solution develops on fitting a human skeleton model to acquired data so as to represent the 3D coordinates of the joints and their change over time as a trajectory in a suitable action space. Thanks to such a 3D joint-based framework, the proposed solution is capable to capture both the shape and the dynamics of the human body simultaneously. The action recognition problem is then formulated as the problem of computing the similarity between the shape of trajectories in a Riemannian manifold. Classification using kNN is finally performed on this manifold taking advantage of Riemannian geometry in the open curve shape space. Experiments are carried out on four representative benchmarks to demonstrate the potential of the proposed solution in terms of accuracy/latency for a low-latency action recognition. Comparative results with state-of-the-art methods are reported

Perceiving What Is Reachable Depends on Motor Representations: Evidence from a Transcranial Magnetic Stimulation Study

Background: Visually determining what is reachable in peripersonal space requires information about the egocentric location of objects but also information about the possibilities of action with the body, which are context dependent. The aim of the present study was to test the role of motor representations in the visual perception of peripersonal space. Methodology: Seven healthy participants underwent a TMS study while performing a right-left decision (control) task or perceptually judging whether a visual target was reachable or not with their right hand. An actual grasping movement task was also included. Single pulse TMS was delivered 80 % of the trials on the left motor and premotor cortex and on a control site (the temporo-occipital area), at 90 % of the resting motor threshold and at different SOA conditions (50ms, 100ms, 200ms or 300ms). Principal Findings: Results showed a facilitation effect of the TMS on reaction times in all tasks, whatever the site stimulated and until 200ms after stimulus presentation. However, the facilitation effect was on average 34ms lower when stimulating the motor cortex in the perceptual judgement task, especially for stimuli located at the boundary of peripersonal space. Conclusion: This study provides the first evidence that brain motor area participate in the visual determination of what is reachable. We discuss how motor representations may feed the perceptual system with information about possibl

Visualizing the Effects of rTMS in a Patient Sample: Small N vs. Group Level Analysis

The use of transcranial magnetic stimulation (TMS) to assess changes in cortical excitability is a tool used with increased prevalence in healthy and impaired populations. One factor of concern with this technique is how to achieve adequate statistical power given constraints of a small number of subjects and variability in responses. This paper compares a single pulse excitability measure using traditional group-level statistics vs single subject analyses in a patient population of subjects with focal hand dystonia, pre and post repetitive TMS (rTMS). Results show significant differences in cortical excitability for 4/5 subjects using a split middle line analysis on plots of individual subject data. Group level statistics (ANOVA), however, did not detect any significant findings. The consideration of single subject statistics for TMS excitability measures may assist researchers in describing the variably of rTMS outcome measures

The TMS Map Scales with Increased Stimulation Intensity and Muscle Activation

One way to study cortical organisation, or its reorganisation, is to use transcranial magnetic stimulation (TMS) to construct a map of corticospinal excitability. TMS maps are reported to be acquired with a wide variety of stimulation intensities and levels of muscle activation. Whilst MEPs are known to increase both with stimulation intensity and muscle activation, it remains to be established what the effect of these factors is on the map's centre of gravity (COG), area, volume and shape. Therefore, the objective of this study was to systematically examine the effect of stimulation intensity and muscle activation on these four key map outcome measures. In a first experiment, maps were acquired with a stimulation intensity of 110, 120 and 130% of resting threshold. In a second experiment, maps were acquired at rest and at 5, 10, 20 and 40% of maximum voluntary contraction. Map area and map volume increased with both stimulation intensity (P 0.09 in all cases). This result indicates the map simply scales with stimulation intensity and muscle activation

Repeatability of Corticospinal and Spinal Measures during Lengthening and Shortening Contractions in the Human Tibialis Anterior Muscle

Elements of the human central nervous system (CNS) constantly oscillate. In addition, there are also methodological factors and changes in muscle mechanics during dynamic muscle contractions that threaten the stability and consistency of transcranial magnetic stimulation (TMS) and perpherial nerve stimulation (PNS) measures

Systematic assessment of training-induced changes in corticospinal output to hand using frameless stereotaxic transcranial magnetic stimulation.

Measuring changes in the characteristics of corticospinal output has become a critical part of assessing the impact of motor experience on cortical organization in both the intact and injured human brain. In this protocol we describe a method for systematically assessing training-induced changes in corticospinal output that integrates volumetric anatomical MRI with transcranial magnetic stimulation (TMS). A TMS coil is sited to a target grid superimposed onto a 3D MRI of cortex using a stereotaxic neuronavigation system. Subjects are then required to exercise the first dorsal interosseus (FDI) muscle on two different tasks for a total of 30 min. The protocol allows for reliably and repeatedly detecting changes in corticospinal output to FDI muscle in response to brief periods of motor training

Mirror Symmetric Bimanual Movement Priming Can Increase Corticomotor Excitability and Enhance Motor Learning

Repetitive mirror symmetric bilateral upper limb may be a suitable priming technique for upper limb rehabilitation after stroke. Here we demonstrate neurophysiological and behavioural after-effects in healthy participants after priming with 20 minutes of repetitive active-passive bimanual wrist flexion and extension in a mirror symmetric pattern with respect to the body midline (MIR) compared to an control priming condition with alternating flexion-extension (ALT). Transcranial magnetic stimulation (TMS) indicated that corticomotor excitability (CME) of the passive hemisphere remained elevated compared to baseline for at least 30 minutes after MIR but not ALT, evidenced by an increase in the size of motor evoked potentials in ECR and FCR. Short and long-latency intracortical inhibition (SICI, LICI), short afferent inhibition (SAI) and interhemispheric inhibition (IHI) were also examined using pairs of stimuli. LICI differed between patterns, with less LICI after MIR compared with ALT, and an effect of pattern on IHI, with reduced IHI in passive FCR 15 minutes after MIR compared with ALT and baseline. There was no effect of pattern on SAI or FCR H-reflex. Similarly, SICI remained unchanged after 20 minutes of MIR. We then had participants complete a timed manual dexterity motor learning task with the passive hand during, immediately after, and 24 hours after MIR or control priming. The rate of task completion was faster with MIR priming compared to control conditions. Finally, ECR and FCR MEPs were examined within a pre-movement facilitation paradigm of wrist extension before and after MIR. ECR, but not FCR, MEPs were consistently facilitated before and after MIR, demonstrating no degradation of selective muscle activation. In summary, mirror symmetric active-passive bimanual movement increases CME and can enhance motor learning without degradation of muscle selectivity. These findings rationalise the use of mirror symmetric bimanual movement as a priming modality in post-stroke upper limb rehabilitation

Muscles in “Concert”: Study of Primary Motor Cortex Upper Limb Functional Topography

BACKGROUND: Previous studies with Transcranial Magnetic Stimulation (TMS) have focused on the cortical representation of limited group of muscles. No attempts have been carried out so far to get simultaneous recordings from hand, forearm and arm with TMS in order to disentangle a 'functional' map providing information on the rules orchestrating muscle coupling and overlap. The aim of the present study is to disentangle functional associations between 12 upper limb muscles using two measures: cortical overlapping and cortical covariation of each pair of muscles. Interhemispheric differences and the influence of posture were evaluated as well. METHODOLOGY/PRINCIPAL FINDINGS: TMS mapping studies of 12 muscles belonging to hand, forearm and arm were performed. Findings demonstrate significant differences between the 66 pairs of muscles in terms of cortical overlapping: extremely high for hand-forearm muscles and very low for arm vs hand/forearm muscles. When right and left hemispheres were compared, overlapping between all possible pairs of muscles in the left hemisphere (62.5%) was significantly higher than in the right one (53.5% ). The arm/hand posture influenced both measures of cortical association, the effect of Position being significant [p = .021] on overlapping, resulting in 59.5% with prone vs 53.2% with supine hand, but only for pairs of muscles belonging to hand and forearm, while no changes occurred in the overlapping of proximal muscles with those of more distal districts. CONCLUSIONS/SIGNIFICANCE: Larger overlapping in the left hemisphere could be related to its lifetime higher training of all twelve muscles studied with respect to the right hemisphere, resulting in larger intra-cortical connectivity within primary motor cortex. Altogether, findings with prone hand might be ascribed to mechanisms facilitating coupling of muscles for object grasping and lifting -with more proximal involvement for joint stabilization- compared to supine hand facilitating actions like catching. TMS multiple-muscle mapping studies permit a better understanding of motor control and 'plastic' reorganization of motor system

Directing visual attention during action observation modulates corticospinal excitability

Transcranial magnetic stimulation (TMS) research has shown that corticospinal excitability is facilitated during the observation of human movement. However, the relationship between corticospinal excitability and participants’ visual attention during action observation is rarely considered. Nineteen participants took part in four conditions: (i) a static hand condition, involving observation of a right hand holding a ball between the thumb and index finger; (ii) a free observation condition, involving observation of the ball being pinched between thumb and index finger; and (iii and iv) finger-focused and ball-focused conditions, involving observation of the same ball pinch action with instructions to focus visual attention on either the index finger or the ball. Single-pulse TMS was delivered to the left motor cortex and motor evoked potentials (MEPs) were recorded from the first dorsal interosseous (FDI) and abductor digiti minimi muscles of the right hand. Eye movements were recorded simultaneously throughout each condition. The ball-focused condition produced MEPs of significantly larger amplitude in the FDI muscle, compared to the free observation or static hand conditions. Furthermore, regression analysis indicated that the number of fixations on the ball was a significant predictor of MEP amplitude in the ball-focused condition. These results have important implications for the design and delivery of action observation interventions in motor (re)learning settings. Specifically, providing viewing instructions that direct participants to focus visual attention on task-relevant objects affected by the observed movement promotes activity in the motor system in a more optimal manner than free observation or no instructions