Perception of co-speech gestures in aphasic patients: A visual exploration study during the observation of dyadic conversations

Co-speech gestures are part of nonverbal communication during conversations. They either support the verbal message or provide the interlocutor with additional information. Furthermore, they prompt as nonverbal cues the cooperative process of turn taking. In the present study, we investigated the influence of co-speech gestures on the perception of dyadic dialogue in aphasic patients. In particular, we analysed the impact of co-speech gestures on gaze direction (towards speaker or listener) and fixation of body parts. We hypothesized that aphasic patients, who are restricted in verbal comprehension, adapt their visual exploration strategies.Methods: Sixteen aphasic patients and 23 healthy control subjects participated in the study. Visual exploration behaviour was measured by means of a contact-free infrared eye-tracker while subjects were watching videos depicting spontaneous dialogues between two individuals. Cumulative fixation duration and mean fixation duration were calculated for the factors co-speech gesture (present and absent), gaze direction (to the speaker or to the listener), and region of interest (ROI), including hands, face, and body.Results: Both aphasic patients and healthy controls mainly fixated the speaker's face. We found a significant co-speech gesture × ROI interaction, indicating that the presence of a co-speech gesture encouraged subjects to look at the speaker. Further, there was a significant gaze direction × ROI × group interaction revealing that aphasic patients showed reduced cumulative fixation duration on the speaker's face compared to healthy controls.Conclusion: Co-speech gestures guide the observer's attention towards the speaker, the source of semantic input. It is discussed whether an underlying semantic processing deficit or a deficit to integrate audio-visual information may cause aphasic patients to explore less the speaker's face

The modulation of reading strategies by language opacity in early bilinguals: an eye movement study

Converging evidences from eye movement experiments indicate that linguistic contexts influence reading strategies. However, the question of whether different linguistic contexts modulate eye movements during reading in the same bilingual individuals remains unresolved. We examined reading strategies in a transparent (German) and an opaque (French) language of early, highly proficient French–German bilinguals: participants read aloud isolated French and German words and pseudo- words while the First Fixation Location (FFL), its duration and latency were measured. Since transparent linguistic contexts and pseudo-words would favour a direct grapheme/phoneme conversion, the reading strategy should be more local for German than for French words (FFL closer to the beginning) and no difference is expected in pseudo-words’ FFL between contexts. Our results confirm these hypotheses, providing the first evidence that the same individuals engage different reading strategy depending on language opacity, suggesting that a given brain process can be modulated by a given context

The Impact of Language Opacity and Proficiency on Reading Strategies in Bilinguals: An Eye Movement Study

Reading strategies vary across languages according to orthographic depth - the complexity of the grapheme in relation to phoneme conversion rules - notably at the level of eye movement patterns. We recently demonstrated that a group of early bilinguals, who learned both languages equally under the age of seven, presented a first fixation location (FFL) closer to the beginning of words when reading in German as compared with French. Since German is known to be orthographically more transparent than French, this suggested that different strategies were being engaged depending on the orthographic depth of the used language. Opaque languages induce a global reading strategy, and transparent languages force a local/serial strategy. Thus, pseudo-words were processed using a local strategy in both languages, suggesting that the link between word forms and their lexical representation may also play a role in selecting a specific strategy. In order to test whether corresponding effects appear in late bilinguals with low proficiency in their second language (L2), we present a new study in which we recorded eye movements while two groups of late German-French and French-German bilinguals read aloud isolated French and German words and pseudo-words. Since, a transparent reading strategy is local and serial, with a high number of fixations per stimuli, and the level of the bilingual participants' L2 is low, the impact of language opacity should be observed in L1. We therefore predicted a global reading strategy if the bilinguals' L1 was French (FFL close to the middle of the stimuli with fewer fixations per stimuli) and a local and serial reading strategy if it was German. Thus, the L2 of each group, as well as pseudo-words, should also require a local and serial reading strategy. Our results confirmed these hypotheses, suggesting that global word processing is only achieved by bilinguals with an opaque L1 when reading in an opaque language; the low level in the L2 gives way to a local and serial reading strategy. These findings stress the fact that reading behavior is influenced not only by the linguistic mode but also by top-down factors, such as readers' proficiency

Multimodal communication in aphasia: perception and production of co-speech gestures during face-to-face conversation

The role of nonverbal communication in patients with post-stroke language impairment (aphasia) is not yet fully understood. This study investigated how aphasic patients perceive and produce co-speech gestures during face-to-face interaction, and whether distinct brain lesions would predict the frequency of spontaneous co-speech gesturing. For this purpose, we recorded samples of conversations in patients with aphasia and healthy participants. Gesture perception was assessed by means of a head-mounted eye-tracking system, and the produced co-speech gestures were coded according to a linguistic classification system. The main results are that meaning-laden gestures (e.g., iconic gestures representing object shapes) are more likely to attract visual attention than meaningless hand movements, and that patients with aphasia are more likely to fixate co-speech gestures overall than healthy participants. This implies that patients with aphasia may benefit from the multimodal information provided by co-speech gestures. On the level of co-speech gesture production, we found that patients with damage to the anterior part of the arcuate fasciculus showed a higher frequency of meaning-laden gestures. This area lies in close vicinity to the premotor cortex and is considered to be important for speech production. This may suggest that the use of meaning-laden gestures depends on the integrity of patients’ speech production abilities

Comprehension of co-speech gestures in aphasic patients: an eye movement study

Co-speech gestures are omnipresent and a crucial element of human interaction by facilitating language comprehension. However, it is unclear whether gestures also support language comprehension in aphasic patients. Using visual exploration behavior analysis, the present study aimed to investigate the influence of congruence between speech and co-speech gestures on comprehension in terms of accuracy in a decision task.Method: Twenty aphasic patients and 30 healthy controls watched videos in which speech was either combined with meaningless (baseline condition), congruent, or incongruent gestures. Comprehension was assessed with a decision task, while remote eye-tracking allowed analysis of visual exploration.Results: In aphasic patients, the incongruent condition resulted in a significant decrease of accuracy, while the congruent condition led to a significant increase in accuracy compared to baseline accuracy. In the control group, the incongruent condition resulted in a decrease in accuracy, while the congruent condition did not significantly increase the accuracy. Visual exploration analysis showed that patients fixated significantly less on the face and tended to fixate more on the gesturing hands compared to controls.Conclusion: Co-speech gestures play an important role for aphasic patients as they modulate comprehension. Incongruent gestures evoke significant interference and deteriorate patients’ comprehension. In contrast, congruent gestures enhance comprehension in aphasic patients, which might be valuable for clinical and therapeutic purposes

Acute cognitively engaging exergame-based physical activity enhances executive functions in adolescents

The study aimed to elucidate the influence of cognitive engagement comprised in an acute bout of exergame-based physical activity on executive functions (inhibition, cognitive flexibility) in adolescents. Therefore, the level of cognitive engagement and the intensity of physical activity were systematically varied across three experimental conditions. Sixty-five healthy male adolescents (13–16 years) were randomly assigned to one of three conditions: (a) physical activity with high levels of cognitive engagement during active video gaming, (b) physical activity with low levels of cognitive engagement during active video gaming, (c) sedentary with low levels of cognitive engagement during passive video watching. Manipulation checks, including subjective and objective operationalizations of cognitive engagement, were applied. Executive functions were assessed before and after each condition using the D-KEFS design fluency test. Results showed that cognitive engagement, operationalized by subjects’ ratings and heart rate variability, differed between conditions. The physical activity condition with a high level of cognitive engagement resulted in significantly better performance in cognitive flexibility compared to conditions with low levels of cognitive engagement. Regarding benefits for executive functions in male adolescents, the results indicate that acute physical activity with high cognitive engagement could be more efficient than physical activity of the same intensity with low cognitive engagement. Even though further evidence is needed, these results extend previous research and suggest a methodological approach for measuring cognitive engagement

Acute cognitively engaging exergaming improves cognitive flexibility performance in adolescents.

Introduction A wide range of literature demonstrates the beneficial effects of acute physical activity (PA) on executive functions (EFs) in children (Donnelly et al., 2016; Verburgh, Königs, Scherder & Osterlaan, 2014). Qualitative PA characteristics (Pesce, 2012) and in particular cognitive engagement (Schmidt, Benzing & Kamer, 2016) have been proposed as important factors influencing these benefits. However, in adolescents, there is limited and contradictory evidence available on acute effects of cognitive engaging PA on EFs. In general, very few studies (impaired by procedural differences) investigated the impact of cognitive engagement comprised in PA on cognition, whereof a majority did not use an operationalization of cognitive engagement (Tomporowski, McCullick, Pendleton & Pesce, 2015). Therefore, the present study aimed to elucidate the influence of cognitive engagement comprised in an acute bout of exergame-based PA on executive functions (inhibition, cognitive flexibility) in adolescents. Methods Sixty-five healthy male adolescents (M = 14.51; SD = 1.08) were randomly assigned to one of three conditions: (a) PA with high levels of cognitive engagement (Shape Up); (b) PA with low levels of cognitive engagement (Running); (c) sedentary with low levels of cognitive engagement (Control). Manipulation checks, including subjective (rating of perceived cognitive engagement) and objective operationalizations of cognitive engagement (heart rate variability), were applied. EFs were assessed before and after each condition using the D-KEFS design fluency test. Results Manipulation check analyses (ANOVAs) showed that intensity of PA was increased equally in the experimental groups compared to control, whereas cognitive engagement was elevated only in the Shape Up group (ps .05). Discussion/Conclusion Cognitive flexibility was immediately enhanced by acute cognitively engaging PA. Thus, to promote benefits in cognitive performance, these results underline the important role cognitive engagement seems to play in PA. In addition, this study suggests a methodological approach to operationalize cognitive engagement in laboratory and field settings. Both the subjective as well as the objective measure might be a promising tool to investigate the construct of cognitive engagement in future studies. References Donnelly, J. E., Hillman, C. H., Castelli, D., Etnier, J. L., Lee, S., Tomporowski, P., … Szabo-Reed, A. N. (2016). Physical activity, fitness, cognitive function, and academic achievement in children. Medicine & Science in Sports & Exercise, 48(6), 1197–1222. Pesce, C. (2012). Shifting the focus from quantitative to qualitative exercise characteristics in exercise and cognition research. Journal of Sport and Exercise Psychology, 34(6), 766–786. Schmidt, M., Benzing, V., & Kamer, M. (2016). Classroom-based physical activity breaks and children’s Attention: Cognitive engagement works! Frontiers in Psychology, 7, 1–13. Tomporowski, P. D., McCullick, B., Pendleton, D. M., & Pesce, C. (2015). Exercise and children’s cognition: The role of exercise characteristics and a place for metacognition. Journal of Sport and Health Science, 4(1), 47–55

Executive function performance in D-KEFS design fluency test of the experimental conditions.

<p>Executive function performance in D-KEFS design fluency test of the experimental conditions.</p