A handy project : owning, perceiving, and experiencing hands

The purpose of this thesis is to explore the mental representation of hands in the context of owning, perceiving, and experiencing hands. The theoretical knowledge of the thesis is discussed in the first chapter, considering the wider concept of body representation. The second chapter reports the data from a study exploring the relationship between the sense of ownership of hands and motor control. The kinematic features of participants’ hands' movements were recorded using a motion analysis capture system and the associated brain activity was recorded using a near-infrared-spectroscopy device. Thirty-two healthy individuals participated in the study. The data show that the embodiment of a rubber hand through touch without vision does not influence the planning and execution of grasping actions (kinematic and brain imaging data). The third chapter presents a study exploring the influence of disgust on the mental representation of hands and feet in action, or motor imagery (MI). Thirty-six healthy individuals were enrolled (different participants from the study in chapter two). The data show that disgust enhances performance on MI tasks. The fourth chapter further expands the knowledge of hands-related MI abilities by comparing hand-based and foot-based MI tasks on a more implicit and explicit level (i.e. action monitoring required to solve the task). Fifty-five healthy participants participated in the data collection (different participants from the study in chapter three). Data show how differences in the mental representation of hands and feet in action imagery are specific to the degree of monitoring required, occurring only when the task is implicit; in other words, when the degree of action monitoring decreases. In the fifth chapter, a comparison between the data obtained in MI tasks executed in laboratory-based and online-based settings is presented, as the pandemic led to the opportunity to explore this aspect. This comparison shows comparable results between settings. Finally, in the sixth chapter, a general discussion of the thesis is presented, reasoning on the limitations of the various experiments and their impact. With the current thesis, I further expand the knowledge on the mental representation of hands, also considering feet as opposite body districts to hands. More in detail, my findings highlight i) that the application of the somatic RHI does not influence motion planning and execution (i.e. maximum GA) (owning hands), ii) that disgust influences our ability to mentally rotate hands and feet (perceiving hands), iii) and that the mental representation of hands and feet in action presents differences specific to the degree of action monitoring involved in solving the task (experiencing hands). The mental representation of hands is dynamic and can be influenced differently by different factors

Supporting Multitasking: Evaluation of Novel Input and Output Characteristics to Support Primary and Secondary Task Performance Using Situation Awareness and Mental Workload

Driving and flying are examples of tasks in dynamic process control environments where performing secondary tasks poses risks to the user, potentially resulting in detrimental effects on situation awareness (SA) and mental workload (MWL). The predominant theories of multitasking emphasize the potential of using alternate information processing channels to better support multitasking, and hence prompt the exploration of novel interface characteristics that may reduce cognitive and structural interference. This dissertation uses measures of SA and MWL to evaluate the potential of input and output display characteristics found in novel technologies, such as Google Glass, Pebble Smartwatch, and Windows Surface, to support multitasking performance. Three experiments were performed to evaluate the following input and output characteristics: voice input (Experiments 1 and 3), head-up display (Experiment 1), size of display (Experiments 2 and 3), use of discrete tactile signals (Experiment 2), and use of continuously informing tactile signals (Experiment 3). Experiments 1 and 2 altered only interface characteristics associated with secondary task performance, while Experiment 1 also altered interface characteristics associated with primary task performance in the attempt to more effectively redistribute MWL. While Experiments 1 (Texting and Driving with Google Glass) and 2 (Weather Technology Characteristics in General Aviation Cockpits) indicate the potential SA and MWL benefits of using voice input and larger displays for secondary tasks in multitasking settings, Experiment 3 (Supporting Emergency Vehicle Mobile Command Terminal Use While Driving) sheds light on the limitations of these benefits with increasing task complexity. Experiment 1 showed that combining a head-up display with voice input provided additional marginal SA and MWL benefits. Experiment 2 also suggested SA and MWL benefits when using discrete tactile signals to aid in indicating the need for secondary task attentional shifts. Experiment 3 furthered this exploration of tactile signaling by presenting continuously informing vibrations relating to the primary task, exhibiting both the potential benefits of providing continuous information and the potential drawbacks of overreliance on such displays. These findings have the potential to fundamentally change the way users interact with technology by informing the development of and policies surrounding new products using these features

Affective touch in infancy

Social touch is ubiquitous in caregiver-infant interactions. Research on animal models and preterm human infants has shown that touch is critical for a young organism’s physical and psychological growth. However, the role that social interaction through touch plays in the development of typically developing human infants is poorly understood. The research presented in this thesis investigated neural specialization for social touch and the mechanisms through which social touch might promote early development. I focus on a particular type of touch, slow velocity stroking, shown to activate a particular type of skin fibers in human adults, the CT-fibers, and to elicit affective responses (henceforth affective touch). Research presented here investigated cortical activation and autonomic responses to affective touch, during the first year of life. Firstly, in experiments 1 through 4 functional Near Infrared Spectroscopy (fNIRS) was employed to measure haemodynamic responses to affective and non-affective touch over inferior frontal and temporal cortices. Experiments 1, 2 and 3 used three different non-affective stimuli and revealed that specialization to affective touch in key nodes of the social brain has not developed yet in 5 to 7-months-old infants. Results from Experiment 4 suggest that this specialization emerges near the end of the first year of life (10-montholds). Secondly, in experiments 5 and 6 heart rate changes to affective and nonaffective touch were measured in three different age-groups (2, 7 and 9-monthold). Results revealed that infants in neither group displayed differential responses to the touch stimuli. Further, experiment 5 explored whether affective touch modulates visual attention but an effect was not found. Taken together these findings showed that preferential processing of affective touch is not evident during early development, at least when investigating neural and autonomic responses. In all my studies, I strived to present tactile stimuli in the absence of other social cues, thus ensuring that any effects would have been specific to touch. In the final discussion I suggest that the lack of context might have prevented infants from identifying affective touch. I also discuss the possibility that other forms of inter-personal touch, and not CT-targeted touch, may be critical in early human development, and should be investigated in future research

Early cross-modal interactions and adult human visual cortical plasticity revealed by binocular rivalry

In this research binocular rivalry is used as a tool to investigate different aspects of visual and multisensory perception. Several experiments presented here demonstrated that touch specifically interacts with vision during binocular rivalry and that the interaction likely occurs at early stages of visual processing, probably V1 or V2. Another line of research also presented here demonstrated that human adult visual cortex retains an unexpected high degree of experience-dependent plasticity by showing that a brief period of monocular deprivation produced important perceptual consequences on the dynamics of binocular rivalry, reflecting a homeostatic plasticity. In summary, this work shows that binocular rivalry is a powerful tool to investigate different aspects of visual perception and can be used to reveal unexpected properties of early visual cortex

Therapeutic Alliance as Active Inference: The Role of Therapeutic Touch and Biobehavioural Synchrony in Musculoskeletal Care

Touch is recognised as crucial for survival, fostering cooperative communication, accelerating recovery, reducing hospital stays, and promoting overall wellness and the therapeutic alliance. In this hypothesis and theory paper, we present an entwined model that combines touch for alignment and active inference to explain how the brain develops “priors” necessary for the health care provider to engage with the patient effectively. We appeal to active inference to explain the empirically integrative neurophysiological and behavioural mechanisms that underwrite synchronous relationships through touch. Specifically, we offer a formal framework for understanding – and explaining – the role of therapeutic touch and hands-on care in developing a therapeutic alliance and synchrony between health care providers and their patients in musculoskeletal care. We first review the crucial importance of therapeutic touch and its clinical role in facilitating the formation of a solid therapeutic alliance and in regulating allostasis. We then consider how touch is used clinically – to promote cooperative communication, demonstrate empathy, overcome uncertainty, and infer the mental states of others – through the lens of active inference. We conclude that touch plays a crucial role in achieving successful clinical outcomes and adapting previous priors to create intertwined beliefs. The ensuing framework may help healthcare providers in the field of musculoskeletal care to use hands-on care to strengthen the therapeutic alliance, minimise prediction errors (a.k.a., free energy), and thereby promote recovery from physical and psychological impairments

Neurophysiological signatures of the body representation in the brain using Immersive Virtual Reality

[spa] Múltiples estudios han demostrado que es posible sustituir el cuerpo de una persona por uno virtual produciendo ilusiones de posesión y agencia sobre este nuevo cuerpo. Los efectos y trazas de esta sustitución de cuerpos se pueden medir de manera subjetiva, a través del comportamiento del participante o a través de medidas fisiológicas objetivas. Sin embargo, restan por descubrir muchos de los mecanismos neurológicos subyacentes que provocan dichos efectos. Esta tesis pretende medir correlaciones neurofisiológicas de las ilusiones de posesión y agencia, y de esta manera contribuir al mayor conocimiento del funcionamiento de la representación corporal en el cerebro. Para ello realizamos una serie de estudios donde la representación del cuerpo humano y su actividad son externamente manipuladas de diversas maneras mediante estimulación multisensorial mientras se registra la correspondiente actividad cerebral mediante electroencefalografía (EEG). La estimulación multisensorial permite que humanos sanos puedan tener ilusiones de posesión y agencia sobre avatares (como si sus cuerpos hubieran sido perceptualmente sustituidos por cuerpos virtuales). Más en concreto, mediante el uso de Realidad Virtual (RV) los participantes tienen una perspectiva egocéntrica de un cuerpo yuxtapuesto al suyo (a través de cascos de realidad virtual estereoscópicos). A la vez que pueden mover este cuerpo mediante sistemas de rastreo del movimiento en tiempo real produciendo una estimulación visuo-motora síncrona. La combinación de la perspectiva en primera persona con la estimulación visuo-motora da como resultado una ilusión de posesión y control total del cuerpo virtual, a la que nos referimos como encarnación virtual. En este escenario exploramos las implicaciones de estas manipulaciones perceptuales en el cerebro mediante EEG. Planteamos un primer experimento con intención de validar si el cuerpo virtual se puede efectivamente reconocer como un sustituto viable del propio cuerpo a nivel del subconsciente. Tras exponer el cuerpo virtual a estímulos nocivos se halla una activación en la corteza motora de los participantes equivalente a la que sería de esperar en un escenario real. Este resultado-como-real hace indicar que los participantes ciertamente aceptan sus cuerpos virtuales dados como su propio cuerpo. En un segundo experimento se estudian las consecuencias de los mecanismos de agencia que nos proporcionan la consciencia de nuestras acciones corporales y motoras, a través de tecnologías de substitución virtual del cuerpo. Los participantes se someten a disrupciones esporádicas de sus acciones mientras realizan movimientos rápidos con su brazo. En algunas ocasiones, la mano virtual del participante se mueve autónomamente y en dirección contraria a la mano real del participante. Los resultados proporcionan evidencias de procesos neuronales específicos responsables de detectar disrupciones de agencia inducidas externamente. Asimismo, estos procesos neuronales se correlacionan con la ilusión de posesión del cuerpo medida a través de cuestionarios. Este estudio profundiza en las perspectivas actuales de los esquemas de agencia, proponiendo y demostrando la teoría existente de mecanismos de errores re-aferentes y de retro-alimentación que funcionan concurrentemente en el cerebro para detectar disrupciones de agencia. Se propone un tercer experimento para explorar las implicaciones de la apariencia externa del cuerpo virtual a la hora de auto-reconocerse. Diseñamos un experimento en el que los participantes se exponen a caras de avatares que se parecen a ellos, a un familiar o a una persona que no conocen. Los resultados muestran mecanismos subyacentes compartidos de auto-identificación de caras reales y virtuales en la corteza visual. En particular, las trazas neurofisiológicas muestran que las caras virtuales se clasifican igual que las reales (en contraste con lo que ocurre durante la clasificación de otros objetos como coches o flores). Además, la corteza visual diferencia niveles de familiaridad entre las caras virtuales. Contribuyendo así de manera novedosa a entender porque avatares con diferente apariencia tienen un impacto en el comportamiento de los participantes, y por ello siendo útil tanto para el campo de la realidad virtual como para el de la auto-identificación. Globalmente, la investigación de esta tesis explora las trazas cerebrales mediante EEG de la experiencia inmediata de poseer y controlar un cuerpo. Se presentan evidencias empíricas para validar el uso de realidad virtual en la investigación de la representación del cuerpo en el cerebro. Asimismo, a través de varios estudios mostramos novedosas trazas neurofisiológicas relacionadas con las manipulaciones corporales y perceptuales. Los resultados y experimentos se contextualizan mediante una revisión en profundidad de la literatura existente en consciencia, percepción corporal y teorías de posesión y agencia del propio cuerpo.[eng] Several studies have shown that it is possible to substitute a person’s body by a virtual one giving rise to illusions of ownership and agency with respect to the virtual body. The effects and traces of such body substitution can be measured using subjective reporting, behavioural and objective physiological measures. However, considerable work remains for uncovering the underlying neurological mechanisms that trigger those effects. In this thesis we aim to measure neurophysiological correlates of ownership and agency, and by doing so to contribute to a greater understanding of the functioning of the body representation in the brain. In order to address this question we carried out a series of studies where the representation of the human body and activity in which it was engaged were externally manipulated in various ways through multisensory stimulation, while measuring the corresponding electroencephalography (EEG) responses. Through this multisensory stimulation, healthy humans experienced full body ownership and agency illusions over virtual bodies – as if their real bodies were perceptually substituted by these bodies. Specifically, Virtual Reality (VR) was used to give participants an egocentric view of a co-located virtual body, using a Head-Mounted Display. They could move this body through real-time motion tracking thus providing synchronous visuo-motor stimulation. The combination of the first person perspective and synchronous visuo-motor stimulation resulted in full body ownership and agency illusions over the virtual body, which we refer to as virtual embodiment. Under these circumstances we then explored the implications of such bodily perceptual manipulations in the brain with EEG. A first study was intended to validate whether a virtual body can be effectively recognized as a feasible substitute to the self-body at the unconscious level. A motor cortex activation equivalent to what would be expected in a real scenario was found after exposing participants’ virtual body to harmful stimuli. This particular result provides a measure of response-as-if-real indicating that participants tended to accept their given virtual bodies as their own. In a second study, the consequences of the agency mechanisms that provide the sensation of control over our own body actions were explored through virtual embodiment. In this study participants underwent sporadic agency disruptions while performing rapid arm movements. In certain conditions, the virtual hand of participants moved autonomously in the opposite direction to the participants’ real hand. Results provide evidence of specific neural processes responsible for detecting externally induced agency disruptions. Moreover, these neural processes were correlated with the strength of the subjective embodiment illusion. This study was also aimed at widening current perspectives on agency schemas, proposing and demonstrating the existing theory of re-afferent and feedback error mechanisms that are concurrently functioning in the brain to detect agency disruptions. In a third study the implications of the external appearance of the substitute virtual body for self-recognition were explored. We designed an experiment in which healthy participants were exposed to self, familiar and unknown faces of look-alike avatars. Results showed shared underlying mechanisms for self-identification in real and virtual faces in the visual cortex. In particular, neurophysiological traces showed that virtual faces are classified as real faces – in contrast to what happens with the classification of other objects (cars, flowers, etc.). Furthermore, the visual cortex differentiated familiarity levels among virtual faces. These are novel insights contributing to the better understanding of why different looking avatars can have an impact on participants’ performance or behaviour, thus being useful both for the fields of virtual embodiment and of self-recognition. Overall the research in this thesis explores brain activity through EEG of the immediate experience of having and controlling a body. Empirical evidence is presented to validate the use of virtual reality for the research of the body representation in the brain. Additionally, novel neural signatures of the bodily perceptual manipulations are presented through a set of studies. Results are put into context with an in-depth review of literature on self-awareness, body perception, body ownership and agency theories at the beginning of this thesis