The Link between Action and Language: Recent Findings and Future Perspectives

This paper aims to present a critical review of studies focused on embodied cognition and, more specifically, on the relationship between language and action. A critical analysis of studies using methods such as TMS and fMRI will be presented, and results reported by the different studies will be discussed, both theoretically and methodologically. Then, in response to some inconsistency detected by the analysis of literature, Virtual Reality will be presented as a possible answer or enrichment for the study of this topic. Possible future research tracks and application are discussed

THE RELATIONSHIPS BETWEEN LANGUAGE AND ACTION: THE CONTRIBUTION OF VIRTUAL REALITY IN THE DOMAIN OF EMBODIED COGNITION

Il razionale di questo progetto affonda le sue radici nelle recenti teorie che considerano il linguaggio come fondato sull’azione, e quindi strettamente collegato al sistema motorio. Negli ultimi decenni, infatti, la scoperta dei neuroni specchio, prima nella scimmia e poi negli esseri umani, ha portato ad un filone di ricerca spesso denominato “embodied language”. Grazie alle metodiche messe a disposizione dalle neuroscienze, ad oggi sono stati raccolti molti dati sperimentali a favore del legame tra sistema motorio e linguaggio, anche se la natura di questo legame non è del tutto chiara. In questa prospettiva, in aggiunta ai tradizionali strumenti di indagine come la Risonanza Magnetica Funzionale (fMRI) o la Stimolazione Magnetica Transcranica (TMS), anche la Realtà Virtuale (RV), che consente di provare un’”esperienza incorporata”, sembra possa aiutare i ricercatori a far luce sulle questioni ancora aperte. Il presente progetto, quindi, si compone di tre ricerche distinte, ciascuna delle quali pone ad oggetto di indagine una sfaccettatura diversa del complesso fenomeno dell’embodied language. Il primo esperimento è finalizzato ad indagare il ruolo della corteccia motoria primaria nei compiti di comprensione, utilizzando la rTMS; nel secondo esperimento viene introdotta la realtà virtuale per valutare se e come un’azione virtuale, grazie a un processo di simulazione, modula la comprensione di verbi; il terzo studio, infine, usando lo stesso ambiente virtuale del secondo studio, si propone di indagare il ruolo dell’azione virtuale durante l’apprendimento di una lingua straniera.The rational of this project is rooted in the recent theories that consider language as grounded in action, and thus tightly tied to the motor system. In the last decades, the discovery of the mirror neurons in monkeys, and of the correspondent mirror neuron system in humans, led to a new research topic often called “embodied language”. Thanks to the methodics supplied by neuroscience, nowadays a great corpus of experimental data has been collected that support the link between language and motor system, even if the nature of this link is still not completely understood. In this perspective, beyond traditional tools such as Functional Magnetic Resonance (fMRI) or Transcranial Magnetic Stimulation (TMS), also Virtual reality (RV), which allows to create an embodied experience, seems suitable to shed light on the open questions. The present project, thereby, is structured in three independent researches, each one aiming at investigating one specific facet of the complex phenomenon of embodied language. The first experiment is designed to investigate the role of the primary motor cortex during language comprehension, using rTMS; in the second one, the virtual reality is introduced, in order to test if and how a virtual action, thanks to simulation, modulates verbs comprehension; the third study, finally, using the same virtual environment, aims at examining the role of the virtual action during foreign language learning

Visual recognition of words learned with gestures induces motor resonance in the forearm muscles

Acknowledgements We thank Sabrina Füreder and Edanur Aktan for their contribution to the data collection.Peer reviewedPublisher PD

Reduction in physical activity during Covid-19 lockdowns predicts individual differences in cognitive performance several months after the end of the safety measures

Peer reviewe

Depth of Encoding Through Observed Gestures in Foreign Language Word Learning

Word learning is basic to foreign language acquisition, however time consuming and not always successful. Empirical studies have shown that traditional (visual) word learning can be enhanced by gestures. The gesture benefit has been attributed to depth of encoding. Gestures can lead to depth of encoding because they trigger semantic processing and sensorimotor enrichment of the novel word. However, the neural underpinning of depth of encoding is still unclear. Here, we combined an fMRI and a behavioral study to investigate word encoding online. In the scanner, participants encoded 30 novel words of an artificial language created for experimental purposes and their translation into the subjects\u2019 native language. Participants encoded the words three times: visually, audiovisually, and by additionally observing semantically related gestures performed by an actress. Hemodynamic activity during word encoding revealed the recruitment of cortical areas involved in stimulus processing. In this study, depth of encoding can be spelt out in terms of sensorimotor brain networks that grow larger the more sensory modalities are linked to the novel word. Word retention outside the scanner documented a positive effect of gestures in a free recall test in the short term

New Trends in Episodic Memory Assessment: Immersive 360° Ecological Videos

How best to measure memory in a reliable and valid way has been intensely debated in neuropsychological literature. Specifically, classical neuropsychological tests often fail to predict real-life performance or capture the multifaceted nature of memory function. To solve these issues, there has been a growing emphasis on the use of more ecological memory assessment. In this scenario, several virtual reality based tools have been developed to evaluate memory function. The aim of the current perspective is to discuss critically the possibilities offered for episodic memory assessment by one of the most innovative trends in the technology field, i.e., 360° videos. Immersivity, egocentric view and realism appear to be crucial features of 360° videos enabling them to enhance the ecological validity of classical assessment tools of memory abilities

The use of Biofeedback in Clinical Virtual Reality: The INTREPID Project

Generalized anxiety disorder (GAD) is a psychiatric disorder characterized by a constant and unspecific anxiety that interferes with daily-life activities. Its high prevalence in general population and the severe limitations it causes, point out the necessity to find new efficient strategies to treat it. Together with the cognitive-behavioral treatments, relaxation represents a useful approach for the treatment of GAD, but it has the limitation that it is hard to be learned. The INTREPID project is aimed to implement a new instrument to treat anxiety-related disorders and to test its clinical efficacy in reducing anxiety-related symptoms. The innovation of this approach is the combination of virtual reality and biofeedback, so that the first one is directly modified by the output of the second one. In this way, the patient is made aware of his or her reactions through the modification of some features of the VR environment in real time. Using mental exercises the patient learns to control these physiological parameters and using the feedback provided by the virtual environment is able to gauge his or her success. The supplemental use of portable devices, such as PDA or smart-phones, allows the patient to perform at home, individually and autonomously, the same exercises experienced in therapist's office. The goal is to anchor the learned protocol in a real life context, so enhancing the patients' ability to deal with their symptoms. The expected result is a better and faster learning of relaxation techniques, and thus an increased effectiveness of the treatment if compared with traditional clinical protocols

Grasping Virtual Objects Benefits Lower Aptitude Learners’ Acquisition of Foreign Language Vocabulary

Acknowledgements Special thanks go to the Ars Electronica Centre, Linz, and to co-workers who aided in the study implementation, particularly Christoph Kremer, Erika Mondria, and the staff of the Future Lab. Supported by the Johannes Kepler Open Access Publishing Fund. Funding Open access funding provided by Johannes Kepler University Linz.Peer reviewedPublisher PD