Temporal Evolution of Both Premotor and Motor Cortical Tuning Properties Reflect Changes in Limb Biomechanics

A prevailing theory in the cortical control of limb movement posits that premotor cortex initiates a high-level motor plan that is transformed by the primary motor cortex (MI) into a low-level motor command to be executed. This theory implies that the premotor cortex is shielded from the motor periphery and therefore its activity should not represent the low-level features of movement. Contrary to this theory, we show that both dorsal (PMd) and ventral premotor (PMv) cortices exhibit population-level tuning properties that reflect the biomechanical properties of the periphery similar to those observed in M1. We recorded single-unit activity from M1, PMd, and PMv and characterized their tuning properties while six rhesus macaques performed a reaching task in the horizontal plane. Each area exhibited a bimodal distribution of preferred directions during execution consistent with the known biomechanical anisotropies of the muscles and limb segments. Moreover, these distributions varied in orientation or shape from planning to execution. A network model shows that such population dynamics are linked to a change in biomechanics of the limb as the monkey begins to move, specifically to the state-dependent properties of muscles. We suggest that, like M1, neural populations in PMd and PMv are more directly linked with the motor periphery than previously thought

The dorsal visual stream revisited: Stable circuits or dynamic pathways?

In both macaque and human brain, information regarding visual motion flows from the extrastriate area V6 along two different paths: a dorsolateral one towards areas MT/V5, MST, V3A, and a dorsomedial one towards the visuomotor areas of the superior parietal lobule (V6A, MIP, VIP). The dorsolateral visual stream is involved in many aspects of visual motion analysis, including the recognition of object motion and self motion. The dorsomedial stream uses visual motion information to continuously monitor the spatial location of objects while we are looking and/or moving around, to allow skilled reaching for and grasping of the objects in structured, dynamically changing environments. Grasping activity is present in two areas of the dorsal stream, AIP and V6A. Area AIP is more involved than V6A in object recognition, V6A in encoding vision for action. We suggest that V6A is involved in the fast control of prehension and plays a critical role in biomechanically selecting appropriate postures during reach to grasp behaviors.In everyday life, numerous functional networks, often involving the same cortical areas, are continuously in action in the dorsal visual stream, with each network dynamically activated or inhibited according to the context. The dorsolateral and dorsomedial streams represent only two examples of these networks. Many others streams have been described in the literature, but it is worthwhile noting that the same cortical area, and even the same neurons within an area, are not specific for just one functional property, being part of networks that encode multiple functional aspects. Our proposal is to conceive the cortical streams not as fixed series of interconnected cortical areas in which each area belongs univocally to one stream and is strictly involved in only one function, but as interconnected neuronal networks, often involving the same neurons, that are involved in a number of functional processes and whose activation changes dynamically according to the context

Sensing with the Motor Cortex

The primary motor cortex is a critical node in the network of brain regions responsible for voluntary motor behavior. It has been less appreciated, however, that the motor cortex exhibits sensory responses in a variety of modalities including vision and somatosensation. We review current work that emphasizes the heterogeneity in sensorimotor responses in the motor cortex and focus on its implications for cortical control of movement as well as for brain-machine interface development

Towards population coding principles in the primate premotor and parietal grasping network

As humans, the only way for us to interact with the world around us is by utilizing our highly trained motor system. Therefore, understanding how the brain generates movement is essential to understanding all aspects of human behavior. Despite the importance the motor system, the manner in which the brain prepares and executes movements, especially grasping movements, is still unclear. In this thesis I undertake a number of electrophysiological and computational experiments on macaque monkeys, primates showing similar grasping behavior to humans, to shed light on how grasping movements are planned and executed across distributed brain regions in both parietal and premotor cortices. Through these experiments, I reveal how the use of large-scale electrophysiological recording of hundreds of neurons simultaneously in primates allows the investigation of network computational principles essential for grasping, and I develop a series of analytical techniques for dissecting the large data sets collected from these experiments. In chapter 2.1 I show how large-scale parallel recordings can be leveraged to make behavioral predictions on single trials. The methods used to extract single-trial predictions varied in their performance, but population-based methods provided the most consistent and meaningful interpretation of the data. In addition, the success of these behavioral predictions could be used to make inferences about how areas differ in their contribution to preparation of grasping movements. It was found that while reaction time could be predicted from the population activity of either area, performance was significantly higher using the data from premotor cortex, suggesting that population activity in premotor cortex may have a more direct effect on behavior. In chapter 2.2 I show how preparation and movement intermingle and interact with one another on the continuum between immediate and withheld movement. Our population-based and dimensionality reduction techniques enable interpretation of the data, even when single neuron tuning properties are highly temporally and functionally complex. Activity in parietal cortex stabilizes during the memory period, while it continues to evolve in premotor cortex, revealing a decodable signature of time. Furthermore, activity during movement initiation clusters into two groups, movements initiated as fast as possible and movements from memory, showing how a state shift likely occurs on the border between these two types of actions. In chapter 2.3 I show that the question of how motor cortex controls movement is an ongoing issue in the field. I address crucial details about recent methodology used to extract rotational dynamics in motor cortex. I show how a simple neural network simulation and novel statistical test reveal properties of motor cortex not examined before, showing how models of movement generation can be essential tools in adding perspective to empirical results. Finally, in chapter 2.4 I show how the specificity of hand use can be used as a tool to dissociate levels of abstraction in the visual to motor transformation in parietal and premotor cortex. While preparatory activity is mostly hand-invariant in parietal cortex, activity in premotor cortex dissociates the intended hand use well before movement. Importantly, we show how appropriate dimensionality reduction techniques can disentangle the effects of multiple task parameters and find latent dimensions consistent between areas and animals. Together, the results of my experiments reinforce the importance of seeing the motor system not as a collection of individually tuned neurons, but as a dynamic network of neurons continuously acting together to produce the complex and flexible behavior we observe in all primates

Neural coding of grasp force planning and control in macaque areas AIP, F5, and M1

In den letzte Jahrzehnten wurde viel daran geforscht zu entschlüsseln wie das Gehirn Greifbewegungen koordiniert. Das anteriore intraparietale Areal (AIP), das Hand Areal des ventralen premotorischen Kortex (F5), und das Hand Areal des primären motorischen Kortex (M1) wurden als essentielle kortikale Arealen für die Kontrolle der Hand identifiziert. Nichtsdestotrotz ist deutlich weniger darüber bekannt wie die Neuronen dieser Areale einen weiteren essentielle Parameter von Greifbewegungen kodieren: Greifkraft. Insbesondere die Rolle der tertiären, kortikalen Areale AIP und F5 in diesen Prozess ist bisher unklar. Die hier durchgeführte Studie befasst sich mit der Wissenslücke über die neuronale Kodierung von Greifkraft Planung und Steuerung in diesen Arealen. Um dies zu erreichen, haben wir zwei Makaken (Macaca mulatta) trainiert eine verzögerte Greifaufgabe auszuführen mit zwei Grifftypen (ein Griff mit der ganzen Hand oder ein Präzisionsgriff) und mit drei verschiedene Kraftniveaus (0-12 N). Während die Affen die Aufgabe ausführten, haben wir die Aktivität von “single-units“ (einzelnen Neuronen) und “multi-units“ (Gruppen von mehreren Neuronen) in den Arealen AIP, F5 und M1 aufgenommen. Wir berechneten den Prozentsatz von Grifftyp modulierten und Griffkraft modulierten “units“ (cluster-based permutation test) und berechneten wie viel Varianz in der Population von “units“ durch Grifftyp und Kraft erklärbar ist, separat für jedes Gehirn Areal mit einer modernen Dimensionalitätsreduktionsanalyse (demixed principal component analysis). 18 Wir zeigen hier zum ersten Mal die Modulation von einzelnen AIP Neuronen durch Greifkraft. Weiterhin bestätigen und erweitern wir hier vorherige Ergebnisse, welche solche neuronale Modulationen bereits in F5 und M1 gezeigt haben. Überaschenderweise war der Prozentsatz von “units“ welche durch Griffkraft moduliert werden, in AIP und F5 nicht wesentlich kleiner als in M1 und ähnlich zu dem Prozentsatz an Grifftyp modulierte Neuronen. Der Anteil an erklärte Varianz in F5 durch Greifkraft war nahezu so groß, wie der Anteil erklärt durch Grifftyp. In AIP und M1 war klar mehr Varianz durch Grifftyp erklärt als durch Kraft, aber der Anteil an erklärte Varianz beider Arealen war ausreichend, um zuverlässig Kraftbedingung zu dekodieren. Wir fanden ebenfalls eine starke neuronale Modulation für Griffkraftbedingungen vor der Bewegungsinitiierung in F5, was wahrscheinlich eine Rolle dieses Areals in der Greifkraftplanung repräsentiert. In AIP war Greifkraftplanungsaktivität nur in einen der beiden Affen vorhanden und wie erwartet nicht präsent in M1 (gemessen nur in einen Affen). Letztendlich, obwohl Greifkraftmodulation in einigen Fällen durch Grifftypmodulation beeinflusst war, war nur ein kleiner Anteil der Populationsvarianz, in den jeweiligen Arealen, durch interaktive Modulation erklärt. Information über Greifkraft können somit folglich separat vom Grifftyp extrahiert werden. Diese Ergebnisse legen eine wichtige Rolle von AIP und F5 bei der Greifkraftkontrolle, neben M1, nah. F5 ist mit hoher Wahrscheinlichkeit auch bei der Planung von Greifkraft involviert, während die Rolle von AIP und M1 geringer ist in diesem Prozess. Letztendlich, da Grifftyp- und Kraftinformation separat extrahierbar sind, zeigen diese Ergebnisse, dass Greifkraft vermutlich unabhängig von Grifftyp, im kortikalen Greifnetzwerk kodiert ist.In de laatste decennia is er veel onderzoek gedaan om te interpreteren hoe de hersenen grijpbewegingen besturen. Het anterieure intra pariëtale gebied (AIP), het handgebied van de ventrale premotorische schors (F5) en het handgebied van de primaire motorische schors (M1) zijn geïdentificeerd als essentiële gebieden van de hersenschors die de vorm van de hand besturen. Maar er is veel minder bekend over hoe de hersenen een andere parameter van grijpbewegingen bestuurt: grijpkracht. Vooral de rol in dit proces van AIP en F5, gebieden van hogere orde, is nog nagenoeg onbekend. Deze studie richt zich op het gebrek aan kennis over de neurale codering van het plannen en besturen van grijpkracht. Om dit te bereiken, hebben we twee makaken (Macaca mulatta) getraind om een vertraagde grijptaak uit te voeren met twee grepen van de hand (een grip met de hele hand of een precisie grip) en met drie verschillende krachtniveaus (0-12 N). Terwijl de apen de taak uitvoerden, maten we de activiteit van single-units (individuele neuronen) en multiunits (collectie van enkele neuronen) in de gebieden AIP, F5 en M1. We berekenden het percentage van units die hun activiteit moduleerden op basis van grip vorm of kracht met een moderne statistieke test (cluster-based permutation test) en we berekenden de hoeveelheid variantie die werd verklaard door de grip vorm en kracht door de populatie van units van elk hersengebied met een moderne dimensie vermindering techniek (demixed principal component analysis). We laten hier voor het eerst zien dat individuele neuronen van AIP hun activiteit moduleren op basis van grijpkracht. Verder bevestigen we dat neuronen van F5 en M1 20 dergelijke modulaties vertonen en breiden we de kennis hierover uit. Verassend genoeg was het percentage units dat reageert op het besturen van grijpkracht in AIP en F5 niet veel lager dan in M1 en ongeveer gelijk aan de hoeveelheid units dat reageert op grip vorm. De hoeveelheid variantie die werd verklaard door grijpkracht in F5 was bijna net zo hoog als wat werd verklaard door grip vorm. In AIP en M1 verklaarde grip vorm duidelijk meer variantie dan grijpkracht, maar ook in deze gebieden was de hoeveelheid variantie dat grijpkracht verklaarde hoog genoeg om de kracht conditie te decoderen. We vonden ook een sterke neurale modulatie voor grijpkracht condities in F5 voordat de arm bewoog, wat mogelijk een rol voor dit gebied representeert in het plannen van grijpkracht. In AIP was activiteit voor het plannen van grijpkracht alleen in één van beide apen gevonden en zoals verwacht was het niet gevonden in M1 (onderzocht in één aap). Tenslotte vonden we dat, hoewel modulatie voor kracht werd beïnvloedt door grip vorm in sommige eenheden, slechts een kleine fractie van de variantie van de neurale populatie van elk hersengebied een gemixte selectiviteit voor grip vorm en kracht had. Informatie over grijpkracht kon daarom onafhankelijk van grip vorm worden geëxtraheerd. Deze bevindingen suggereren een belangrijke rol voor AIP en F5 in het besturen van grijpkracht, samen met M1. F5 is waarschijnlijk ook betrokken met het plannen van grijpkracht, terwijl AIP en M1 waarschijnlijk een kleinere rol hebben in dit proces. Tenslotte, omdat informatie over grip vorm en grijpkracht onafhankelijk konden worden geëxtraheerd, laten deze resultaten zien dat grijpkracht vermoedelijk onafhankelijk van hand vorm is gecodeerd in het grijpnetwerk van de hersenschors

Differential neural dynamics underling pragmatic and semantic affordance processing in macaque ventral premotor cortex

Premotor neurons play a fundamental role in transforming physical properties of observed objects, such as size and shape, into motor plans for grasping them, hence contributing to "pragmatic" affordance processing. Premotor neurons can also contribute to "semantic" affordance processing, as they can discharge differently even to pragmatically identical objects depending on their behavioural relevance for the observer (i.e. edible or inedible objects). Here, we compared the response of monkey ventral premotor area F5 neurons tested during pragmatic (PT) or semantic (ST) visuomotor tasks. Object presentation responses in ST showed shorter latency and lower object selectivity than in PT. Furthermore, we found a difference between a transient representation of semantic affordances and a sustained representation of pragmatic affordances at both the single neuron and population level. Indeed, responses in ST returned to baseline within 0.5 s whereas in PT they showed the typical sustained visual-to-motor activity during Go trials. In contrast, during No-go trials, the time course of pragmatic and semantic information processing was similar. These findings suggest that premotor cortex generates different dynamics depending on pragmatic and semantic information provided by the context in which the to-be-grasped object is presented

The Influence of the Dorsal Pathway on Enhanced Visual Processing

Overall our visual experience is such a seamless one that unless specifically told, we might never know that what we see is actually the visual system taking the very simple input provided by cells in the retina and constructing an image based on rules and calculations and algorithms neuroscientists have yet to fully uncover. This is an incredible feat given the plethora of visual stimuli within our environment, that this information is used to inform and plan actions, and if that wasnt enough, the visual system also has the capacity to selectively enhance certain aspects of visual processing if needs be. The research contained within this dissertation seeks to investigate how the dorsal visual pathway enhances both decision-making processes and visual stimuli presented near the hand. Our findings suggest that the formation of object representations in the dorsal pathway can include both ventral (colour, contrast) and dorsal (speed) stream features (chapters two and three), which in turn greatly speed decision-making processes within the dorsal pathway. In addition, contrast and speed are integrated automatically but purely ventral stream features, such as colour, require top-down attention to facilitate enhanced processing speeds (chapter three). In chapter four we find that visual processing near the hand is enhanced in a novel way. When the hand is nearby, orientation tuning is sharpened in a manner not consistent with either oculomotor-driven spatial or feature based attention. In addition, response variability is reduced when the hand is nearby, raising the possibility that enhanced processing near the hand maybe be driven by feedback from frontoparietal reaching and grasping regions. The research within this dissertation includes important new information regarding how the dorsal pathway can speed visual processing, and provides insight as to the next stage in understanding how we use vision for action

The neuroscience of vision-based grasping: a functional review for computational modeling and bio-inspired robotics

The topic of vision-based grasping is being widely studied using various techniques and with different goals in humans and in other primates. The fundamental related findings are reviewed in this paper, with the aim of providing researchers from different fields, including intelligent robotics and neural computation, a comprehensive but accessible view on the subject. A detailed description of the principal sensorimotor processes and the brain areas involved in them is provided following a functional perspective, in order to make this survey especially useful for computational modeling and bio-inspired robotic application

The cognitive neuroscience of prehension: recent developments

Prehension, the capacity to reach and grasp, is the key behavior that allows humans to change their environment. It continues to serve as a remarkable experimental test case for probing the cognitive architecture of goal-oriented action. This review focuses on recent experimental evidence that enhances or modifies how we might conceptualize the neural substrates of prehension. Emphasis is placed on studies that consider how precision grasps are selected and transformed into motor commands. Then, the mechanisms that extract action relevant information from vision and touch are considered. These include consideration of how parallel perceptual networks within parietal cortex, along with the ventral stream, are connected and share information to achieve common motor goals. On-line control of grasping action is discussed within a state estimation framework. The review ends with a consideration about how prehension fits within larger action repertoires that solve more complex goals and the possible cortical architectures needed to organize these actions

Influence of Gaze Position on Grasp Parameters For Reaches to Visible and Remembered Stimuli

In order to pick up or manipulate a seen object, one must use visual signals to aim and transport the hand to the object’s location (reach), and configure the digits to the shape of the object (grasp). It has been shown that reach and grasp are controlled by separate neural pathways. In real world conditions, however, all of these signals (gaze, reach, grasp) must interact to provide accurate eye-hand coordination. The interactions between gaze, reach, and grasp parameters have not been comprehensively studied in humans. The purpose of the study was to investigate 1) the effect of gaze and target positions on grasp location, amplitude, and orientation, and 2) the influence of visual feedback of the hand and target on the final grasp components and on the spatial deviations associated with gaze direction and target position. Seven subjects reached to grasp a rectangular “virtual” target presented at three orientations, three locations, and with three gaze fixation positions during open- and closed-loop conditions. Participants showed gaze- and target-dependent deviations in grasp parameters that could not be predicted from previous studies. Our results showed that both reach- and grasp-related deviations were affected by stimulus position. The interaction effects of gaze and reach position revealed complex mechanisms, and their impacts were different in each grasp parameter. The impacts of gaze direction on grasp deviation were dependent on target position in space, especially for grasp location and amplitude. Gaze direction had little impact on grasp orientation. Visual feedback about the hand and target modulated the reach- and gaze- related impacts. The results suggest that the brain uses both control signal interactions and sensorimotor strategies to control and plan reach-and-grasp movements