Perception of co-speech gestures in aphasic patients: A visual exploration study during the observation of dyadic conversations

Co-speech gestures are part of nonverbal communication during conversations. They either support the verbal message or provide the interlocutor with additional information. Furthermore, they prompt as nonverbal cues the cooperative process of turn taking. In the present study, we investigated the influence of co-speech gestures on the perception of dyadic dialogue in aphasic patients. In particular, we analysed the impact of co-speech gestures on gaze direction (towards speaker or listener) and fixation of body parts. We hypothesized that aphasic patients, who are restricted in verbal comprehension, adapt their visual exploration strategies.Methods: Sixteen aphasic patients and 23 healthy control subjects participated in the study. Visual exploration behaviour was measured by means of a contact-free infrared eye-tracker while subjects were watching videos depicting spontaneous dialogues between two individuals. Cumulative fixation duration and mean fixation duration were calculated for the factors co-speech gesture (present and absent), gaze direction (to the speaker or to the listener), and region of interest (ROI), including hands, face, and body.Results: Both aphasic patients and healthy controls mainly fixated the speaker's face. We found a significant co-speech gesture × ROI interaction, indicating that the presence of a co-speech gesture encouraged subjects to look at the speaker. Further, there was a significant gaze direction × ROI × group interaction revealing that aphasic patients showed reduced cumulative fixation duration on the speaker's face compared to healthy controls.Conclusion: Co-speech gestures guide the observer's attention towards the speaker, the source of semantic input. It is discussed whether an underlying semantic processing deficit or a deficit to integrate audio-visual information may cause aphasic patients to explore less the speaker's face

Comprehension of co-speech gestures in aphasic patients: an eye movement study

Co-speech gestures are omnipresent and a crucial element of human interaction by facilitating language comprehension. However, it is unclear whether gestures also support language comprehension in aphasic patients. Using visual exploration behavior analysis, the present study aimed to investigate the influence of congruence between speech and co-speech gestures on comprehension in terms of accuracy in a decision task.Method: Twenty aphasic patients and 30 healthy controls watched videos in which speech was either combined with meaningless (baseline condition), congruent, or incongruent gestures. Comprehension was assessed with a decision task, while remote eye-tracking allowed analysis of visual exploration.Results: In aphasic patients, the incongruent condition resulted in a significant decrease of accuracy, while the congruent condition led to a significant increase in accuracy compared to baseline accuracy. In the control group, the incongruent condition resulted in a decrease in accuracy, while the congruent condition did not significantly increase the accuracy. Visual exploration analysis showed that patients fixated significantly less on the face and tended to fixate more on the gesturing hands compared to controls.Conclusion: Co-speech gestures play an important role for aphasic patients as they modulate comprehension. Incongruent gestures evoke significant interference and deteriorate patients’ comprehension. In contrast, congruent gestures enhance comprehension in aphasic patients, which might be valuable for clinical and therapeutic purposes

Theta burst stimulation improves overt visual search in spatial neglect independently of attentional load

Visual neglect is considerably exacerbated by increases in visual attentional load. These detrimental effects of attentional load are hypothesised to be dependent on an interplay between dysfunctional inter-hemispheric inhibitory dynamics and load-related modulation of activity in cortical areas such as the posterior parietal cortex (PPC). Continuous Theta Burst Stimulation (cTBS) over the contralesional PPC reduces neglect severity. It is unknown, however, whether such positive effects also operate in the presence of the detrimental effects of heightened attentional load. Here, we examined the effects of cTBS on neglect severity in overt visual search (i.e., with eye movements), as a function of high and low visual attentional load conditions. Performance was assessed on the basis of target detection rates and eye movements, in a computerised visual search task and in two paper-pencil tasks. cTBS significantly ameliorated target detection performance, independently of attentional load. These ameliorative effects were significantly larger in the high than the low load condition, thereby equating target detection across both conditions. Eye movement analyses revealed that the improvements were mediated by a redeployment of visual fixations to the contralesional visual field. These findings represent a substantive advance, because cTBS led to an unprecedented amelioration of overt search efficiency that was independent of visual attentional load

Enhancing treatment effects by combining continuous theta burst stimulation with smooth pursuit training

Continuous theta burst stimulation (cTBS) represents a promising approach in the treatment of neglect syndrom. However, it is not known whether cTBS in conjunction with another technique may enhance the therapeutic effects. In the present sham-controlled study, we aimed to combine cTBS with smooth pursuit training (SPT), another method known to effectively improve neglect symptoms, and to evaluate whether this combination would result in a stronger effect than SPT alone. Eighteen patients with left spatial neglect after right-hemispheric stroke were included in the study and performed a cancellation task on a large 54.6" touchscreen monitor. A sequential application of cTBS and SPT induced a significantly greater improvement of neglect than SPT alone. After the combined application of these two methods, patients detected significantly more targets and their cancellation behaviour presented a significantly greater shift towards the contralesional hemispace. We suggest that a combined, sequential application of cTBS and SPT is a promising new approach to treat neglect

Contralesional Trunk Rotation Dissociates Real vs. Pseudo-Visual Field Defects due to Visual Neglect in Stroke Patients

In stroke patients, the clinical presentation of visual field defects (VFDs) is frequently accompanied by visual neglect, i.e., the inability to attend and respond to the contralesional space. However, the diagnostic discrimination between the lack of reactions to contralesional stimuli due to VFDs or visual neglect is challenging during clinical examination. This discrimination is particularly relevant, since both clinical pictures are associated with different therapeutic approaches and outcomes. The aim of this study was to systematically investigate the effectiveness of trunk rotation toward the contralesional side-a manipulation dissociating the coordinate system of the trunk from that of the head and eyes-in disentangling real VFDs from "pseudo-VFDs" that occur due to visual neglect. Twenty patients with a left-sided VFD after a right-hemispheric stroke (10 additionally showing visual neglect in neuropsychological testing, VFD + neglect; 10 without neglect, VFD) were tested with Goldmann perimetry in both standard and trunk rotation conditions. In the standard condition, both VFD and VFD + neglect patients showed a conspicuous narrowing of the left visual field. However, trunk rotation triggered strikingly different patterns of change in the two groups: it elicited a significant increase in visual field extension in the VFD + neglect group, but left visual field extension virtually unchanged in the VFD group. Our results highlight contralesional trunk rotation as a simple, viable manipulation to effectively and rapidly disentangle real VFDs from "pseudo-VFDs" (i.e., due to visual neglect) during clinical examination

Neglect and Motion Stimuli - Insights from a Touchscreen-Based Cancellation Task

BACKGROUND AND PURPOSE: In stroke patients, neglect diagnostic is often performed by means of paper-pencil cancellation tasks. These tasks entail static stimuli, and provide no information concerning possible changes in the severity of neglect symptoms when patients are confronted with motion. We therefore aimed to directly contrast the cancellation behaviour of neglect patients under static and dynamic conditions. Since visual field deficits often occur in neglect patients, we analysed whether the integrity of the optic radiation would influence cancellation behaviour. METHODS: Twenty-five patients with left spatial neglect after right-hemispheric stroke were tested with a touchscreen cancellation task, once when the evenly distributed targets were stationary, and once when the identic targets moved with constant speed on a random path. The integrity of the right optic radiation was analysed by means of a hodologic probabilistic approach. RESULTS: Motion influenced the cancellation behaviour of neglect patients, and the direction of this influence (i.e., an increase or decrease of neglect severity) was modulated by the integrity of the right optic radiation. In patients with an intact optic radiation, the severity of neglect significantly decreased in the dynamic condition. Conversely, in patients with damage to the optic radiation, the severity of neglect significantly increased in the dynamic condition. CONCLUSION: Motion may influence neglect in stroke patients. The integrity of the optic radiation may be a predictor of whether motion increases or decreases the severity of neglect symptoms

Different visual exploration of tool-related gestures in left hemisphere brain damaged patients is associated with poor gestural imitation.

According to the direct matching hypothesis, perceived movements automatically activate existing motor components through matching of the perceived gesture and its execution. The aim of the present study was to test the direct matching hypothesis by assessing whether visual exploration behavior correlate with deficits in gestural imitation in left hemisphere damaged (LHD) patients. Eighteen LHD patients and twenty healthy control subjects took part in the study. Gesture imitation performance was measured by the test for upper limb apraxia (TULIA). Visual exploration behavior was measured by an infrared eye-tracking system. Short videos including forty gestures (20 meaningless and 20 communicative gestures) were presented. Cumulative fixation duration was measured in different regions of interest (ROIs), namely the face, the gesturing hand, the body, and the surrounding environment. Compared to healthy subjects, patients fixated significantly less the ROIs comprising the face and the gesturing hand during the exploration of emblematic and tool-related gestures. Moreover, visual exploration of tool-related gestures significantly correlated with tool-related imitation as measured by TULIA in LHD patients. Patients and controls did not differ in the visual exploration of meaningless gestures, and no significant relationships were found between visual exploration behavior and the imitation of emblematic and meaningless gestures in TULIA. The present study thus suggests that altered visual exploration may lead to disturbed imitation of tool related gestures, however not of emblematic and meaningless gestures. Consequently, our findings partially support the direct matching hypothesis

Eye Gaze Behavior at Turn Transition: How Aphasic Patients Process Speakers' Turns during Video Observation

The human turn-taking system regulates the smooth and precise exchange of speaking turns during face-to-face interaction. Recent studies investigated the processing of ongoing turns during conversation by measuring the eye movements of noninvolved observers. The findings suggest that humans shift their gaze in anticipation to the next speaker before the start of the next turn. Moreover, there is evidence that the ability to timely detect turn transitions mainly relies on the lexico-syntactic content provided by the conversation. Consequently, patients with aphasia, who often experience deficits in both semantic and syntactic processing, might encounter difficulties to detect and timely shift their gaze at turn transitions. To test this assumption, we presented video vignettes of natural conversations to aphasic patients and healthy controls, while their eye movements were measured. The frequency and latency of event-related gaze shifts, with respect to the end of the current turn in the videos, were compared between the two groups. Our results suggest that, compared with healthy controls, aphasic patients have a reduced probability to shift their gaze at turn transitions but do not show significantly increased gaze shift latencies. In healthy controls, but not in aphasic patients, the probability to shift the gaze at turn transition was increased when the video content of the current turn had a higher lexico-syntactic complexity. Furthermore, the results from voxel-based lesion symptom mapping indicate that the association between lexico-syntactic complexity and gaze shift latency in aphasic patients is predicted by brain lesions located in the posterior branch of the left arcuate fasciculus. Higher lexico-syntactic processing demands seem to lead to a reduced gaze shift probability in aphasic patients. This finding may represent missed opportunities for patients to place their contributions during everyday conversation