Bodily form, actions and motor skills’ neural representations: Transcranial Magnetic Stimulation studies.

Recent years have seen the development of new neurophysiological techniques that deepened the knowledge of neural processes underlying simple as well as complex brain mechanisms of behaviour and cognitive processes. Knowledge about body and motor representations in the brain has advanced much through the discovery of mirror neurons in monkeys and their putative homologous counterpart in humans. Simulative-like mechanisms have been proposed to be at the base of perceptive and cognitive functions. The present work focuses on neural underpinnings of visual body and action perception and motor skills representation in the motor system. These issues are at the core of plastic neural changes in action learning, understanding and higher cognitive functions as abstract motor reasoning and identity categorization. This thesis reports three studies in which these domains have been studied through single- pulse TMS and high-frequency event-related repetitive TMS procedures

To what extent does object knowledge bias the perception of goal-directed actions?

Predictive processing accounts of action understanding suggest that inferred goals generate top-down predictions that bias perception towards expected goals. These predictions are thought to be derived, in part, from the affordances of available objects. This thesis had three aims: (1) to test whether high-level action goals based on object knowledge can bias action perception, (2) to investigate the degree to which this perceptual bias can be influenced by high-level person knowledge, or by expertise in particular objects, (3) to explore the low-level mechanisms underlying the anticipatory representation of action goals associated with objects. Experiments used a modified representational momentum paradigm, as well as RT-based measures. In Chapter 2, we found that the presentation of a prime object led to a predictive bias in the perception of a subsequent action towards a functionally related target object. This bias was present for reaching actions, but not withdrawing actions (Experiment 1a) and persisted even when the functionally related target was simultaneously presented with an unrelated distractor (Experiment 1b). Crucially, this effect was specific to intentional actions, but was eliminated when the hand was replaced by a non-biological object following the same trajectory. This finding supports predictive processing views that action perception is guided by goal predictions, based on prior knowledge about the context in which the action occurs. We found no evidence that this perceptual bias could be influenced by prior knowledge about the gender of the actor (Chapter 3) or by participants' expertise in particular objects (Chapter 4). Chapter 5 tested for motor biases resulting from object-based goal predictions. Originally designed as a TMS study (Experiment 4a), this was tested online using RT measures as an index for motor preparation. We found no evidence that object affordances can be reliably measured using online RTs. Taken together these findings highlight the important role of object knowledge in action perception, while showing the limits to which this might be modulated by person knowledge and expertise. The final chapter highlights the challenges of developing robust behavioural measures for online testing of object affordances

Electrocortical underpinnings of error monitoring in health and pathology

It becomes clear from the literature described above (Chapter 1), that the error monitoring mechanisms play a fundamental role in signalling the need for cognitive control. Many studies already provided a consistent evidence on the existence of peculiar ways in which the brain signals this need through electrophysiological changes. However, the following set of empirical studies aims to gain further insight into these complex processes by measuring brain activity changes in situations that alter the way one experience errors. The second Chapter (Chapter 2) consists of a brief commentary that was made in response to an article on the brain activity to action errors. In this commentary we propose new possibilities to explore our topic of interest, by taking advantage of EEG and modern virtual reality facilities. The thesis includes three EEG-VR studies: one on the error-mechanism in healthy participants (Chapter 3) and two studies on error monitoring system in pathological populations (Chapter 4, 5), as main parts of the core of the thesis. As a collateral project, in the Appendix, there is an EEG study on action observation in elite players (Chapter 7). In the first study (Chapter 3), we investigated a very simple but fundamental question. As we saw in the introduction, error-related signatures are evoked when an error occurs. But it is not clear how much of this is due to the occurrence of a violation of the intended goal or simply to the observation of a rare – thus less predictable – event. To this aim, we used a paradigm developed in the former years in our laboratory (Pavone et al., 2016; Spinelli et al., 2017), characterized by a setup in immersive Virtual Reality (VR) and simultaneous EEG recording. Building on the previous findings, we designed an EEG-VR study in which we manipulated the probability of observing errors in actions. In another study (Chapter 4) we investigated how erroneous actions are experienced by people with brain damage and diagnosis of Apraxia. Apraxic patients are people with hemispheric lesions and defective awareness on a variety of aspects that cover perceptuo-motor, cognitive or emotional domains. This study was developed after the results obtained by Canzano and colleagues (2014) in a behavioral study in which apraxic patients were asked to imitate the actions executed by the experimenter and judge their correctness; results revealed that bucco-facial apraxic patients manifest a specific deficit in detecting their own gestural errors when they are explicitly asked to judge them. With the present study we wanted to investigate apraxic brain’ response to action errors, while they embody an avatar in first person perspective (EEG-VR setup). The third study (Chapter 5) investigates the integrity of the error-monitoring system in Parkinson’s Disease and the impact of the dopaminergic treatment in the brain response to errors. To this aim we used the proposed VR action-observation paradigm, in which Parkinson patients observed successful and unsuccessful reach-to-grasp actions in first person perspective while EEG activity was recorded; the same patients were tested while being under dopaminergic treatment and during a dopaminergic withdrawal state. In another chapter we provide a critical overview of the findings of this work (General Discussion, Chapter 6). In the last chapter, the Appendix (Chapter 7), there is a collateral project of another research line of the Laboratory, in which I have being involved. In this study we are investigating the cortical underpinning of elite players during observation of goal-directed actions, in their domain of expertise. We recorded the EEG activity of elite wheelchair basketball players while observing free-throws performed by paraplegic athletes. We expected their brain correlates to be different from novice players and to be able to easily discriminate whether a basketball shot would be successful or unsuccessful (project still ongoing)

Eye quietness and quiet eye in expert and novice golf performance: an electrooculographic analysis

Quiet eye (QE) is the final ocular fixation on the target of an action (e.g., the ball in golf putting). Camerabased eye-tracking studies have consistently found longer QE durations in experts than novices; however, mechanisms underlying QE are not known. To offer a new perspective we examined the feasibility of measuring the QE using electrooculography (EOG) and developed an index to assess ocular activity across time: eye quietness (EQ). Ten expert and ten novice golfers putted 60 balls to a 2.4 m distant hole. Horizontal EOG (2ms resolution) was recorded from two electrodes placed on the outer sides of the eyes. QE duration was measured using a EOG voltage threshold and comprised the sum of the pre-movement and post-movement initiation components. EQ was computed as the standard deviation of the EOG in 0.5 s bins from –4 to +2 s, relative to backswing initiation: lower values indicate less movement of the eyes, hence greater quietness. Finally, we measured club-ball address and swing durations. T-tests showed that total QE did not differ between groups (p = .31); however, experts had marginally shorter pre-movement QE (p = .08) and longer post-movement QE (p < .001) than novices. A group × time ANOVA revealed that experts had less EQ before backswing initiation and greater EQ after backswing initiation (p = .002). QE durations were inversely correlated with EQ from –1.5 to 1 s (rs = –.48 - –.90, ps = .03 - .001). Experts had longer swing durations than novices (p = .01) and, importantly, swing durations correlated positively with post-movement QE (r = .52, p = .02) and negatively with EQ from 0.5 to 1s (r = –.63, p = .003). This study demonstrates the feasibility of measuring ocular activity using EOG and validates EQ as an index of ocular activity. Its findings challenge the dominant perspective on QE and provide new evidence that expert-novice differences in ocular activity may reflect differences in the kinematics of how experts and novices execute skills

Decoding Mental States after Severe Brain Injury

Some patients with disorders of consciousness retain sensory and cognitive abilities that are not apparent from their outward behaviour. It is crucial to identify and characterise these covert abilities for diagnosis, prognosis, and medical ethics. This thesis uses neuroimaging techniques to investigate cognitive preservation and awareness in patients who are behaviourally non-responsive due to acquired brain injuries. In the first chapter, a large sample of healthy volunteers, including experienced athletes and musicians, imagined actions of varying complexity and familiarity. Motor imagery involving certain complex, familiar actions correlated with a more robust sensorimotor rhythm. In the second chapter, several patients with disorders of consciousness participated in multiple experiments based on neural responses to mental imagery, including one task featuring complex, familiar imagined actions. Although the patients did not generate enhanced sensorimotor rhythms for the complex, familiar motor imagery, the detection of covert cognition was more sensitive owing to the multi-modal nature of the assessment. In the final empirical chapter, a sample of healthy volunteers and a heterogeneous cohort of patients with disorders of consciousness completed a novel oddball task based on tactile stimulation. Critically, this task delineated an attentional hierarchy in the patient sample, and patients with the ability to follow commands were differentiated from those unable to do so by event-related potential evidence of attentional orienting. Due to the heterogeneity of aetiology and pathology in the disorders of consciousness, these patients vary in their suitability for neuroimaging, the preservation of neural structures, and the cognitive resources available to them. Assessments of several perceptual and cognitive abilities supported by spatially-distinct brain regions and indexed by multiple neural signatures are therefore required to accurately characterise a patient’s abilities and probable subjective experience

Modeling and Learning of Complex Motor Tasks: A Case Study with Robot Table Tennis

Most tasks that humans need to accomplished in their everyday life require certain motor skills. Although most motor skills seem to rely on the same elementary movements, humans are able to accomplish many different tasks. Robots, on the other hand, are still limited to a small number of skills and depend on well-defined environments. Modeling new motor behaviors is therefore an important research area in robotics. Computational models of human motor control are an essential step to construct robotic systems that are able to solve complex tasks in a human inhabited environment. These models can be the key for robust, efficient, and human-like movement plans. In turn, the reproduction of human-like behavior on a robotic system can be also beneficial for computational neuroscientists to verify their hypotheses. Although biomimetic models can be of great help in order to close the gap between human and robot motor abilities, these models are usually limited to the scenarios considered. However, one important property of human motor behavior is the ability to adapt skills to new situations and to learn new motor skills with relatively few trials. Domain-appropriate machine learning techniques, such as supervised and reinforcement learning, have a great potential to enable robotic systems to autonomously learn motor skills. In this thesis, we attempt to model and subsequently learn a complex motor task. As a test case for a complex motor task, we chose robot table tennis throughout this thesis. Table tennis requires a series of time critical movements which have to be selected and adapted according to environmental stimuli as well as the desired targets. We first analyze how humans play table tennis and create a computational model that results in human-like hitting motions on a robot arm. Our focus lies on generating motor behavior capable of adapting to variations and uncertainties in the environmental conditions. We evaluate the resulting biomimetic model both in a physically realistic simulation and on a real anthropomorphic seven degrees of freedom Barrett WAM robot arm. This biomimetic model based purely on analytical methods produces successful hitting motions, but does not feature the flexibility found in human motor behavior. We therefore suggest a new framework that allows a robot to learn cooperative table tennis from and with a human. Here, the robot first learns a set of elementary hitting movements from a human teacher by kinesthetic teach-in, which is compiled into a set of motor primitives. To generalize these movements to a wider range of situations we introduce the mixture of motor primitives algorithm. The resulting motor policy enables the robot to select appropriate motor primitives as well as to generalize between them. Furthermore, it also allows to adapt the selection process of the hitting movements based on the outcome of previous trials. The framework is evaluated both in simulation and on a real Barrett WAM robot. In consecutive experiments, we show that our approach allows the robot to return balls from a ball launcher and furthermore to play table tennis with a human partner. Executing robot movements using a biomimetic or learned approach enables the robot to return balls successfully. However, in motor tasks with a competitive goal such as table tennis, the robot not only needs to return the balls successfully in order to accomplish the task, it also needs an adaptive strategy. Such a higher-level strategy cannot be programed manually as it depends on the opponent and the abilities of the robot. We therefore make a first step towards the goal of acquiring such a strategy and investigate the possibility of inferring strategic information from observing humans playing table tennis. We model table tennis as a Markov decision problem, where the reward function captures the goal of the task as well as knowledge on effective elements of a basic strategy. We show how this reward function, and therefore the strategic information can be discovered with model-free inverse reinforcement learning from human table tennis matches. The approach is evaluated on data collected from players with different playing styles and skill levels. We show that the resulting reward functions are able to capture expert-specific strategic information that allow to distinguish the expert among players with different playing skills as well as different playing styles. To summarize, in this thesis, we have derived a computational model for table tennis that was successfully implemented on a Barrett WAM robot arm and that has proven to produce human-like hitting motions. We also introduced a framework for learning a complex motor task based on a library of demonstrated hitting primitives. To select and generalize these hitting movements we developed the mixture of motor primitives algorithm where the selection process can be adapted online based on the success of the synthesized hitting movements. The setup was tested on a real robot, which showed that the resulting robot table tennis player is able to play a cooperative game against an human opponent. Finally, we could show that it is possible to infer basic strategic information in table tennis from observing matches of human players using model-free inverse reinforcement learning

Virtual Reality Games for Motor Rehabilitation

This paper presents a fuzzy logic based method to track user satisfaction without the need for devices to monitor users physiological conditions. User satisfaction is the key to any product’s acceptance; computer applications and video games provide a unique opportunity to provide a tailored environment for each user to better suit their needs. We have implemented a non-adaptive fuzzy logic model of emotion, based on the emotional component of the Fuzzy Logic Adaptive Model of Emotion (FLAME) proposed by El-Nasr, to estimate player emotion in UnrealTournament 2004. In this paper we describe the implementation of this system and present the results of one of several play tests. Our research contradicts the current literature that suggests physiological measurements are needed. We show that it is possible to use a software only method to estimate user emotion