The effect of continuous, nonlinearly transformed visual feedback on rapid aiming movements

We investigated the ability to adjust to nonlinear transformations that allow people to control external systems like machines and tools. Earlier research (Verwey and Heuer 2007) showed that in the presence of just terminal feedback participants develop an internal model of such transformations that operates at a relatively early processing level (before or at amplitude specification). In this study, we investigated the level of operation of the internal model after practicing with continuous visual feedback. Participants executed rapid aiming movements, for which a nonlinear relationship existed between the target amplitude seen on the computer screen and the required movement amplitude of the hand on a digitizing tablet. Participants adjusted to the external transformation by developing an internal model. Despite continuous feedback, explicit awareness of the transformation did not develop and the internal model still operated at the same early processing level as with terminal feedback. Thus with rapid aiming movements, the type of feedback may not matter for the locus of operation of the internal model

Exploiting Multiple Sensory Modalities in Brain-Machine Interfaces

Recent improvements in cortically-controlled brain-machine interfaces (BMIs) have raised hopes that such technologies may improve the quality of life of severely motor-disabled patients. However, current generation BMIs do not perform up to their potential due to the neglect of the full range of sensory feedback in their strategies for training and control. Here we confirm that neurons in the primary motor cortex (MI) encode sensory information and demonstrate a significant heterogeneity in their responses with respect to the type of sensory modality available to the subject about a reaching task. We further show using mutual information and directional tuning analyses that the presence of multi-sensory feedback (i.e. vision and proprioception) during replay of movements evokes neural responses in MI that are almost indistinguishable from those responses measured during overt movement. Finally, we suggest how these playback-evoked responses may be used to improve BMI performance

A Production System Model of Capturing Reactive Moving Targets

Subjects manipulated a control stick to position a cursor over a moving target that reacted with a computer-generated escape strategy. The cursor movements were described at two levels of abstraction. At the upper level, a production system described transitions among four modes of activity; rapid acquisition, close following, a predictive mode, and herding. Within each mode, differential equations described trajectory-generating mechanisms. A simulation of this two-level model captures the targets in a manner resembling the episodic time histories of human subjects

Premotor cortex encoding of dynamic hand force and motor output observation underlying hand-object interaction

We studied encoding of hand force and its relations with observation-related activity in macaques trained in an isometric hand-force application and recalibration task, that required to move a visual cursor on a screen toward eight peripheral targets by exerting a force on an isometric joystick, in absence or presence of an opposing force field. Monkeys also observed the result of their action in play-back, as motion of a visual cursor on the screen. This approach combined in a single experiment isometric action performance and force adjustment with observation of its consequences in the external world, also allowing to determine whether PMd neuronal populations reflected an inverse model that specified the force necessary to move in different directions a visual object, or a forward computation encoding its desired trajectory in visual space. We found that a population of PMd cells encoded the direction of dynamic force and its recalibration when the force condition changed but did not retain memory of such change, probably reflecting an adaptation rather than a learning process. Cells with observation-related activity also modulated by change in hand force were not modulated when the force conditions changed, suggesting that their activity reflected the motion of the visual cursor on the screen, therefore the consequences of force application in the visual space. These results also allow a direct comparison of the relative contribute of different populations of PMd cells with that of cells with similar activity profile in the encoding of hand force and its consequences in the parieto-frontal system

Variability in wrist-tilt accelerometer based gesture interfaces

In this paper we describe a study that examines human performance in a tilt control targeting task on a PDA. A three-degree of freedom accelerometer attached to the base of the PDA allows users to navigate to the targets by tilting their wrist in different directions. Post hoc analysis of performance data has been used to classify the ease of targeting and variability of movement in the different directions. The results show that there is an increase in variability of motions upwards from the centre, compared to downwards motions. Also the variability in the x axis component of the motion was greater than that in the y axis. This information can be used to guide designers as to the ease of various relative motions, and can be used to reshape the dynamics of the interaction to make each direction equally easy to achieve

Incorporating Feedback from Multiple Sensory Modalities Enhances Brain–Machine Interface Control

The brain typically uses a rich supply of feedback from multiple sensory modalities to control movement in healthy individuals. In many individuals, these afferent pathways, as well as their efferent counterparts, are compromised by disease or injury resulting in significant impairments and reduced quality of life. Brain–machine interfaces (BMIs) offer the promise of recovered functionality to these individuals by allowing them to control a device using their thoughts. Most current BMI implementations use visual feedback for closed-loop control; however, it has been suggested that the inclusion of additional feedback modalities may lead to improvements in control. We demonstrate for the first time that kinesthetic feedback can be used together with vision to significantly improve control of a cursor driven by neural activity of the primary motor cortex (MI). Using an exoskeletal robot, the monkey\u27s arm was moved to passively follow a cortically controlled visual cursor, thereby providing the monkey with kinesthetic information about the motion of the cursor. When visual and proprioceptive feedback were congruent, both the time to successfully reach a target decreased and the cursor paths became straighter, compared with incongruent feedback conditions. This enhanced performance was accompanied by a significant increase in the amount of movement-related information contained in the spiking activity of neurons in MI. These findings suggest that BMI control can be significantly improved in paralyzed patients with residual kinesthetic sense and provide the groundwork for augmenting cortically controlled BMIs with multiple forms of natural or surrogate sensory feedback

Interregional synchrony of visuomotor tracking: perturbation effects and individual differences

The present study evaluated the neural and behavioural correlates associated with a visuomotor tracking task during which a sensory perturbation was introduced that created a directional bias between moving hand and cursor position. The results revealed that trajectory error increased as a result of the perturbation in conjunction with a dynamic neural reorganization of cluster patterns that reflected distinct processing. In particular, a negatively activated cluster, characterizing the degraded information processing due to the perturbation, involved both hemispheres as well as midline area. Conversely, a positively activated cluster, indicative of compensatory processing was strongly confined to the left (dominant) hemisphere. In addition, a brain-behavioural association of good vs. poor performing participants enabled to localize a neural circuit within the left hemisphere and midline area that linked with successful performance. Overall, these data reinforce the functional significance of interregional synchrony in defining response output and behavioural success