Visual attention and working memory in action

This doctoral thesis employed a psychophysical approach to investigate the relationship between goal-directed eye and hand movements, visual attention, and visual working memory. To establish a solid methodological basis for investigating visual attention, the first study compared the strengths and weaknesses of a set of discrimination stimuli frequently used in attention research (Chapter 2.1). Based on the results, we used a novel pink noise stimulus for approaching the following research questions concerning visual attention. In the second study, we investigated the dependence of attentional orienting on oculomotor programming (Chapter 2.2). Motivated by the claim that attention can only be allocated to locations reachable by saccadic eye movements, we measured visual sensitivity – a proxy for visual attention – within and beyond the oculomotor range using an eye abduction paradigm. Contrary to previous findings, we found that attention can be shifted without restriction to locations to which saccades cannot be executed, ruling out the necessity to program a saccadic eye movement as a prerequisite for spatial attention. The third study attempted to resolve the longstanding debate as to whether eye and hand movement targets are selected by a single attentional mechanism or by independent, effector-specific systems (Chapter 2.3). Results revealed that during simultaneous eye and hand movements, attention – an index of motor target selection – was allocated in parallel to the saccade and the reach targets. Motor target selection mechanisms moreover did not compete for attentional resources at any time during movement preparation, demonstrating that separate, effector-specific mechanisms attentionally select eye and hand movement targets. The fourth study tested the assumption of effector-specific selection mechanisms in the framework of visual working memory (Chapter 2.4). Participants memorized several locations and performed eye, hand, or simultaneous eye-hand movements during the maintenance interval. When participants performed an eye and a hand movement simultaneously to distinct locations, memory at both motor targets was enhanced with no tradeoff between the two. This suggests that the two effector systems improve working memory at their selected motor targets independently. In the final study, we dissociated the relative contributions of the two highly interdependent parameters, task relevance and oculomotor selection, to the memory benefits consistently observed at eye movement targets (Chapter 2.5). Participants memorized shapes while simultaneously either avoiding or selecting a specific location as a delayed saccade target. While oculomotor selection was consistently associated with an increased working memory performance, mere task relevance was not, indicating that the frequently reported memory benefits for task-relevant items might, in fact, be caused by oculomotor selection. In summary, goal-directed eye and hand movements selectively boost the visual processing of the currently most relevant information, and likewise bias our memory capacities according to behavioral priority. The observed motor-induced enhancements in both the attention and working memory domains appear to be independent and effector-specific, allowing for the most flexible assignment of our limited cognitive resources as we traverse through our crowded environment

Independent selection of eye and hand targets suggests effector-specific attentional mechanisms

Both eye and hand movements bind visual attention to their target locations during movement preparation. However, it remains contentious whether eye and hand targets are selected jointly by a single selection system, or individually by independent systems. To unravel the controversy, we investigated the deployment of visual attention - a proxy of motor target selection - in coordinated eye-hand movements. Results show that attention builds up in parallel both at the eye and the hand target. Importantly, the allocation of attention to one effector's motor target was not affected by the concurrent preparation of the other effector's movement at any time during movement preparation. This demonstrates that eye and hand targets are represented in separate, effector-specific maps of action-relevant locations. The eye-hand synchronisation that is frequently observed on the behavioral level must emerge from mutual influences of the two effector systems at later, post-attentional processing stages

Tool use modulates early stages of visuo-tactile integration in far space:Evidence from event-related potentials

The neural representation of multisensory space near the body is modulated by the active use of long tools in non-human primates. Here, we investigated whether the electrophysiological correlates of visuo-tactile integration in near and far space were modulated by active tool use in healthy humans. Participants responded to a tactile target delivered to one hand while an irrelevant visual stimulus was presented ipsilaterally in near or far space. This crossmodal task was performed after the use of either short or long tools. Crucially, the P100 components elicited by visuo-tactile stimuli was enhanced on far as compared to near space trials after the use of long tools, while no such difference was present after short tool use. Thus, we found increased neural responses in brain areas encoding tactile stimuli to the body when visual stimuli were presented close to the tip of the tool after long tool use. This increased visuo-tactile integration on far space trials following the use of long tools might indicate a transient remapping of multisensory space. We speculate that performing voluntary actions with long tools strengthens the representation of sensory information arising within portions of space (i.e. the hand and the tip of the tool) that are most functionally relevant to one's behavioural goals

Visual attention and working memory in action

This doctoral thesis employed a psychophysical approach to investigate the relationship between goal-directed eye and hand movements, visual attention, and visual working memory. To establish a solid methodological basis for investigating visual attention, the first study compared the strengths and weaknesses of a set of discrimination stimuli frequently used in attention research (Chapter 2.1). Based on the results, we used a novel pink noise stimulus for approaching the following research questions concerning visual attention. In the second study, we investigated the dependence of attentional orienting on oculomotor programming (Chapter 2.2). Motivated by the claim that attention can only be allocated to locations reachable by saccadic eye movements, we measured visual sensitivity – a proxy for visual attention – within and beyond the oculomotor range using an eye abduction paradigm. Contrary to previous findings, we found that attention can be shifted without restriction to locations to which saccades cannot be executed, ruling out the necessity to program a saccadic eye movement as a prerequisite for spatial attention. The third study attempted to resolve the longstanding debate as to whether eye and hand movement targets are selected by a single attentional mechanism or by independent, effector-specific systems (Chapter 2.3). Results revealed that during simultaneous eye and hand movements, attention – an index of motor target selection – was allocated in parallel to the saccade and the reach targets. Motor target selection mechanisms moreover did not compete for attentional resources at any time during movement preparation, demonstrating that separate, effector-specific mechanisms attentionally select eye and hand movement targets. The fourth study tested the assumption of effector-specific selection mechanisms in the framework of visual working memory (Chapter 2.4). Participants memorized several locations and performed eye, hand, or simultaneous eye-hand movements during the maintenance interval. When participants performed an eye and a hand movement simultaneously to distinct locations, memory at both motor targets was enhanced with no tradeoff between the two. This suggests that the two effector systems improve working memory at their selected motor targets independently. In the final study, we dissociated the relative contributions of the two highly interdependent parameters, task relevance and oculomotor selection, to the memory benefits consistently observed at eye movement targets (Chapter 2.5). Participants memorized shapes while simultaneously either avoiding or selecting a specific location as a delayed saccade target. While oculomotor selection was consistently associated with an increased working memory performance, mere task relevance was not, indicating that the frequently reported memory benefits for task-relevant items might, in fact, be caused by oculomotor selection. In summary, goal-directed eye and hand movements selectively boost the visual processing of the currently most relevant information, and likewise bias our memory capacities according to behavioral priority. The observed motor-induced enhancements in both the attention and working memory domains appear to be independent and effector-specific, allowing for the most flexible assignment of our limited cognitive resources as we traverse through our crowded environment

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development

Cognition in Sensorimotor Control: Interfacing With the Posterior Parietal Cortex

Millions of people worldwide are afflicted with paralysis from a disruption of neural pathways between the brain and the muscles. Because their cortical architecture is often preserved, these patients are able to plan movements despite an inability to execute them. In such people, brain machine interfaces have great potential to restore lost function through neuroprosthetic devices, circumventing dysfunctional corticospinal circuitry. These devices have typically derived control signals from the motor cortex (M1) which provides information highly correlated with desired movement trajectories. However, sensorimotor control simultaneously engages multiple cognitive processes such as intent, state estimation, decision making, and the integration of multisensory feedback. As such, cortical association regions upstream of M1 such as the posterior parietal cortex (PPC) that are involved in higher order behaviors such as planning and learning, rather than in encoding movement itself, may enable enhanced, cognitive control of neuroprosthetics, termed cognitive neural prosthetics (CNPs). We illustrate in this review, through a small sampling, the cognitive functions encoded in the PPC and discuss their neural representation in the context of their relevance to motor neuroprosthetics. We aim to highlight through examples a role for cortical signals from the PPC in developing CNPs, and to inspire future avenues for exploration in their research and development

An ERP investigation of the consequences of motor preparation on visual processing.

Action and perception have traditionally be studied in isolation, as separate and unitary cognitive processes. More recent evidence has demonstrated a lively interaction between the two. Preparing an action – either a saccade or a manual movement - causes enhanced processing of action-relevant stimuli in the environment, at the expense of the action-irrelevant. The aim of the research reported in this thesis are to provide further detail about this effect. The experiments are reported in this thesis are concerned with how the spatial, temporal and functional properties of action affect perception. Chapter three reports an experiment in which the spatial properties of a grasped object, which make different demands of accuracy, were manipulated. The experiment in chapter four compared goal and effector locations, and measured visual processing across the time course of motor preparation. Chapter five reports an experiment that measured visual processing not just at goal and effector locations, but also at more distant locations not involved in movement, in order to estimate the spatial profile of the effect. Results showed clear enhancement of goal and effector locations simultaneously during motor preparation, although the goal location was enhanced over a broader time period than the effector, suggesting the two components of movement are not equivalent in terms of the relative priorities assigned to them during motor preparation. The spatial profile of the effect fell-off with distance from the goal and effector, and is discussed in terms of theories concerning the spatial profile of visual attention. Taken together, the results of all three experiments suggest that the processes initiated by motor preparation cause shifts in the patterns of perceptual facilitation and inhibition that ultimately achieve selectivity. The inclusion of the effector location in this process suggests that it is not limited to one representation at a time, but operates instead as a flexible and dynamic rebalancing of perception that adapts to any given cognitive task