Common and Segregated Processing of Observed Actions in Human SPL

To clarify the functional organization of parietal cortex involved in action observation, we scanned subjects observing 3 widely different classes of actions: Manipulation with the hands, locomotion, and climbing. An effector-based organization predicts that parietal regions involved in the observation of climbing should not differ from those involved in observing manipulation and locomotion, opposite to the prediction of an organization based upon the action performed. Compared with individual controls, the observation of climbing evoked activity in dorsal superior parietal lobule (SPL), extending into precuneus and posterior cingulate sulcus. Observation of locomotion differentially activated similar regions less strongly. Observation of manipulation activated ventro-rostral SPL, including putative human AIP (phAIP). Using interaction testing and exclusive masking to directly compare the parietal regions involved in observing the 3 action classes, relative to the controls, revealed that the rostral part of dorsal SPL was specifically involved in observing climbing and phAIP in observing manipulation. Parietal regions common to observing all 3 action classes were restricted and likely reflected higher order visual processing of body posture and 3D structure from motion. These results support a functional organization of some parietal regions involved in action observation according to the type of action in the case of climbing and manipulation

Neuronale Korrelate des Lernens von komplexen Bewegungen

The recognition of complex movements is a fundamental function of our everyday life. Results from object recognition, biological motion processing and theoretical modeling indicate that learning could play an important role in shaping the processing of complex movements in the human brain. The aim of this thesis is to systematically investigate visual learning mechanisms, focusing on three main questions: 1. Are humans able to learn to discriminate between very similar complex movement patterns and what are the possible constraints that influence the learning process? 2. What are the neural correlates of the learning process in the different areas of the visual cortex and is there a difference between the learning of human like movements compared to artificial articulated movement patterns at the level of the BOLD response? 3. Is it be possible to implement biologically plausible learning mechanisms into an already existing model for biological movement recognition and can the model be used to simulate the BOLD activity changes obtained from the functional imaging experiments in order to test different hypothesis about the underlying plasticity mechanisms? According to the three main research questions, the thesis is divided into three experimental chapters. Chapter 3 reports a series of psychophysical experiments that investigated whether humans are able to learn to differentiate between complex movement patterns. These patterns belonged to three different groups of movements and were all presented as point-light animations. The first group was composed of natural human movements, the second one consists of movements of artificial skeleton models containing nine segments moving in an articulated fashion and the third one consisted of the same human movements as the first one, but this time the spatial positions of the individual points were scrambled. The main focus of the psychophysical investigations was to identify possible differences in the learning process between the three different groups with respect to the time scale of the learning and the invariance properties of the learned representation. To control for the learning history of the subjects, all stimuli were generated by motion morphing. In this way it was possible to create stimuli that were completely novel to the observer. Additionally, motion morphing by linear combination of prototypical movements allowed to precisely control the spatio-temporal similarity between the individual stimuli. Chapter 4 presents the results of a series of functional imaging experiments that were carried out to identify possible neural correlates of the learning process. Because the stimuli across the conditions that had to be compared were very similar, a special fMRI adaptation paradigm was used to acquire the images. This technique allows to identify possible sub-populations of neurons within the same voxel that contribute to the encoding of different movement stimuli. In addition to the experimental runs, several localizer runs were acquired for every observer to reliably detect the visual areas involved in low-level, mid-level and high-level motion and form processing (namely V1, V2, V3, V3a, VP, V4v, V3b/KO, hMT+/V5, FFA and STSp). By analyzing the fMRI signal separately for each of these areas, it became possible to identify learning processes at all stages of the visual hierarchy. The main focus of the imaging experiments was to pinpoint learning induced neural plasticity mechanisms in the visual cortex and to identify possible differences between the learning of natural human-like movements compared to artificial articulated movement patterns. The final experimental chapter of this thesis deals with the theoretical implementation of the experimental results. The theoretical part was based on an already existing neural model for biological motion recognition, which was extended by the implementation of biologically plausible learning rules. Specific detectors of complex form and optic flow fields were learned automatically along with the temporal order with which these features arise during the movement sequence. Additionally, a neural adaptation mechanism was implemented at the highest level of the model. The goal of the theoretical part was to be able to simulate a whole run of a real fMRI experiment and to determine whether the simulated BOLD responses are in accordance with measured BOLD responses. In the future, this model could then be used to test different hypothesis about how learning shapes the processing of complex movements in the human brain.Die vorliegende Arbeit beschäftigt sich mit der visuellen Wahrnehmung komplexer biologischer Bewegungen. Vorangegangene Arbeiten legen die Vermutung nahe, dass Lernprozesse bei der Verarbeitung von biologischen Bewegungen von entscheidender Bedeutung sind. Die Dissertation bearbeitet im wesentlichen drei verschiedene Fragestellungen: 1. Können Menschen lernen zwischen sehr ähnlichen komplexen biologischen Bewegungen zu unterscheiden? Gibt es Unterschiede zwischen dem Lernen von menschlichen Bewegungen und artifiziellen Bewegungsmustern? 2. Können wir mit Hilfe von funktioneller Magnetresonanztomographie Veränderungen auf neuronaler Ebene nachweisen, die durch den Lernprozess ausgelöst werden? Sind die gleichen Hirnareale beim Lernen von natuerlichen menschlichen Bewegungen und artifiziellen Bewegungsmustern beteiligt? 3. Können wir ein neuronales Netzwerk entwickeln, mit dem die beobachteten Lernprozesse simuliert werden, um die neuronalen Mechanismen besser zu verstehen? Diese Fragestellungen werden mit Hilfe von psychophysikalischen Messmethoden, funktioneller Bildgebung und theoretischer Modellierung genauer untersucht

Human functional magnetic resonance imaging reveals separation and integration of shape and motion cues in biological motion processing

In a series of human functional magnetic resonance imaging experiments, we systematically manipulated point-light stimuli to identify the contributions of the various areas implicated in biological motion processing (for review, see Giese and Poggio, 2003). The first experiment consisted of a 2 x 2 factorial design with global shape and kinematics as factors. In two additional experiments, we investigated the contributions of local opponent motion, the complexity of the portrayed movement and a one-back task to the activation pattern. Experiment 1 revealed a clear separation between shape and motion processing, resulting in two branches of activation. A ventral region, extending from the lateral occipital sulcus to the posterior inferior temporal gyrus, showed a main effect of shape and its extension into the fusiform gyrus also an interaction. The dorsal region, including the posterior inferior temporal sulcus and the posterior superior temporal sulcus (pSTS), showed a main effect of kinematics together with an interaction. Region of interest analysis identified these interaction sites as the extrastriate and fusiform body areas (EBA and FBA). The local opponent motion cue yielded only little activation, limited to the ventral region (experiment 3). Our results suggest that the EBA and the FBA correspond to the initial stages in visual action analysis, in which the performed action is linked to the body of the actor. Moreover, experiment 2 indicates that the body areas are activated automatically even in the absence of a task, whereas other cortical areas like pSTS or frontal regions depend on the complexity of movements or task instructions for their activation.status: publishe

Time-dependent hebbian rules for the learning of templates for visual motion recognition

Experimental evidence suggests that the visual recognition of biological movements is based on learned spatio-temporal templates. Work in computational vision shows that movement recognition can be accomplished by recognizing temporal sequences of form or optic flow patterns. Recurrent neural networks with asymmetric lateral connections are one physiologically plausible way for the encoding of spatio-temporal templates. We demonstrate that time-dependent hebbian plasticity is suitable for establishing the required lateral connectivity patterns. We tested different hebbian plasticity rules and compared their efficiency and stability properties in simulations and by mathematical analysis. We found the most robust behavior for a learning rule that assumes a normalization of the total afferent synaptic connectivity that can be supported by each neuron. Consistent with psychophysical data our model learns the appropriate lateral connections after less than 30 stimulus repetitions. The resulting recurrent neural network shows strong sequence selectivity.status: publishe

Time-dependent hebbian rules for the learning of templates

for visual motion recognitio

A comparative study of common coding for observed and executed actions in human and non-human primates

status: publishe

Visual learning shapes the processing of complex movement stimuli in the human brain

Recognition of actions and complex movements is fundamental for social interactions and action understanding. While the relationship between motor expertise and visual recognition of body movements has received a vast amount of interest, the role of visual learning remains largely unexplored. Combining psychophysics and fMRI experiments, we investigated neural correlates of visual learning of complex movements. Subjects were trained to visually discriminate between very similar complex movement stimuli generated by motion morphing that were either compatible (Experiments 1 and 2) or incompatible (Experiment 3) with human movement execution. Employing an fMRI adaptation paradigm as index of discriminability, we scanned human subjects before and after discrimination training. The results of Experiment 1 revealed three different effects as a consequence of training; 1) Emerging fMRI-selective adaptation in general motion related areas (hMT/V5+, KO/V3b) for the differences between human-like movements. 2) Enhanced of fMRI-selective adaptation already present before training in biological motion related areas (pSTS, FBA). 3) Changes covarying with task difficulty in frontal areas. Moreover, the observed activity changes were specific to the trained movement patterns (Experiment 2). The results of Experiment 3, testing artificial movement stimuli, were strikingly similar to the results obtained for human movements. General and biological motion related areas showed movement-specific changes in fMRI-selective adaptation for the differences between the stimuli after training. These results support the existence of a powerful visual machinery for the learning of complex motion patterns that is independent of motor execution. We propose thus a key role of visual learning in action recognition.status: publishe

Learning to discriminate complex movements: biological versus artificial trajectories

The recognition of complex body movements and actions is a fundamental visual capacity very important for social communication. It seems possible that movement recognition is based on a general capability of the visual system to learn complex visual motion patterns. Alternatively, this visual function might exploit specialized mechanisms for the analysis of biologically relevant movements, for example, of humans or animals. To investigate this question, we trained human observers to discriminate novel motion patterns that were generated, exploiting a new technique for stimulus generation by motion morphing. We tested the learning of different classes of novel movement stimuli. One group of stimuli was fully consistent with human movements. A second class of stimuli was based on artificial skeleton models that were inconsistent with human and animal bodies. A third group of stimuli specified the same local motion information as human movements but was inconsistent with an underlying articulated shape. Participants learned both classes of articulated movements very fast in an orientation-dependent manner. Learning speed and accuracy were strikingly similar and independent of the similarity of the stimuli with biologically relevant body shapes. For the class of stimuli without underlying articulated shape, however, we did not observe significant improvements of the discrimination performance after training. Our results indicate the existence of a fast visual learning process for complex articulated movement patterns, which likely is relevant for biological motion perception. This process seems to operate independently of the consistency of the patterns with biologically relevant body shapes but seems to require the compatibility of the learned movements with a global underlying shape.status: publishe

Heterogeneous single unit selectivity in an fMRI-defined body-selective patch

Although the visual representation of bodies is essential for reproduction, survival, and social communication, little is known about the mechanisms of body recognition at the single neuron level. Imaging studies showed body-category selective regions in the primate occipitotemporal cortex, but it is difficult to infer the stimulus selectivities of the neurons from the population activity measured in these fMRI studies. To overcome this, we recorded single unit activity and local field potentials (LFPs) in the middle superior temporal sulcus body patch, defined by fMRI in the same rhesus monkeys. Both the spiking activity, averaged across single neurons, and LFP gamma power in this body patch was greater for bodies (including monkey bodies, human bodies, mammals, and birds) compared with other objects, which fits the fMRI activation. Single neurons responded to a small proportion of body images. Thus, the category selectivity at the population level resulted from averaging responses of a heterogeneous population of single units. Despite such strong within-category selectivity at the single unit level, two distinct clusters, bodies and nonbodies, were present when analyzing the responses at the population level, and a classifier that was trained using the responses to a subset of images was able to classify novel images of bodies with high accuracy. The body-patch neurons showed strong selectivity for individual body parts at different orientations. Overall, these data suggest that single units in the fMRI-defined body patch are biased to prefer bodies over nonbody objects, including faces, with a strong selectivity for individual body images.status: publishe

Stimulus representations in body-selective regions of the macaque cortex assessed with event-related fMRI

Functional imaging studies in humans and monkeys have shown category-selective regions in the temporal cortex, in particular for faces and bodies. Although the body-selective regions have been well studied in humans, little is understood about the functional properties of such regions in macaques. To address this, we first mapped body-selective activations in the visual cortex of four rhesus monkeys in a block design fMRI study and identified two regions in the middle and anterior Superior Temporal Sulcus (STS) that were more strongly activated by monkey bodies compared to well-controlled manmade objects. These two regions partially overlapped with regions that were more activated by faces than manmade objects. Secondly, using an event-related, single image fMRI design we measured the activations to 200 images of 10 stimulus classes (monkey bodies, human bodies, mammals, birds, monkey faces, human faces, body-like sculptures, fruits/vegetables, and two sets of control objects). Multivoxel-pattern analyses showed that both body-selective regions primarily distinguished faces from other inanimate and animate objects, including bodies. Another distinction was present between inanimate objects and bodies in the middle STS body region. The category-based clustering was less pronounced in the anterior compared to the middle STS body-selective regions. In addition, both body-selective regions showed further selectivity for different "subclasses" of the broad body category such as monkeys, human, mammals and birds. Overall, these data indicate strong spatial clustering of animate categories in the macaque STS with a surprisingly marked distinction between faces and bodies within body-selective regions which was stronger than between manmade objects and bodies.publisher: Elsevier articletitle: Stimulus representations in body-selective regions of the macaque cortex assessed with event-related fMRI journaltitle: NeuroImage articlelink: http://dx.doi.org/10.1016/j.neuroimage.2012.07.013 content_type: article copyright: Copyright © 2012 Elsevier Inc. All rights reserved.status: publishe