The impact of auditory-visual speech perception on working memory

Adding visual speech information (i.e. lip movements) to auditory speech information (i.e. voice) can enhance speech comprehension in younger and older adults while at the same time it reduces electrical brain responses, as measured by event-related potentials (ERPs). Thus, the brain seems to allocate fewer resources to speech comprehension when audio-visual (AV) speech information is available. This study examined whether the brain resources saved at the perceptual level during AV presentation allow younger and older adults to perform better on a working memory task, and whether older adults benefit to the same extent as younger adults. Twenty older adults and 23 younger adults completed an n-back working memory task (0-, 1-,2-, 3-back) under visual-only (V-only), auditory-only (A-only), and AV condition while ERPs were recorded. The results showed a decrease in reaction time across all memory loads and an improvement in accuracy for 2back and 3-back during AV compared to the V-only and A-only conditions. In addition, ERP analysis from a sample of 12 younger and 12 older adults showed a smaller N1 amplitude for the older group during AV compared to A-only presentation. The attenuation of N1, however, did not correlate with behavioural data. Nor did it show a relationship with changes either in the latency or the amplitude of P3, an ERP that reflects working memory processes. Thus, despite clear behavioural improvements on the working memory task during AV speech presentation, a more direct relationship between facilitation of sensory processing and working memory improvement was not identified

The Facilitation of Perceptual Processing by Auditory-Visual Speech and the Subsequent Effect on Working Memory in Older Adults with Hearing Loss or Cognitive Impairment

It has been proposed in the literature that if too many processing resources need to be devoted to perception then higher-order cognitive functions, such as working memory (WM), may suffer. This effect may be particularly evident in individuals who have restricted processing resources, such as older adults (OA) suffering from hearing loss or cognitive impairment. One possibility to facilitate perception during speech processing is through the auditory-visual (AV) modality. The current research examined whether AV speech perception helps to facilitate perceptual and WM processing in OAs with restricted processing resources. In both studies, participants completed a WM n-back task under different speech modalities: AV, auditory-only, or visual-only (Study 1). Both behavioural and event-related potentials (ERPs) measures were collected during the task. Study 1 examined the effect of AV speech on WM in OAs with hearing impairment compared to normal-hearing OAs. The results showed that AV speech in comparison to auditory-only speech led to facilitated perceptual processing in OAs with hearing impairment, as indicated by ERP responses. The AV modality also led to facilitated WM functioning in both groups, as suggested by ERP responses and behavioural reaction time. A few measures indicated that visual speech cues may have helped OAs with hearing impairment to counteract the demanding auditory processing. Study 2 examined the effect of AV speech on WM in OAs suffering from mild cognitive impairment (MCI) or Alzheimer disease (AD) compared to cognitively healthy OAs. The ERP responses showed that the AV modality compared to the auditory-only modality led to facilitated perceptual and WM processing in both groups. In addition, the behavioural results showed improved accuracy during the WM task for the patient group, and faster reaction time for both the patient group and the cognitively healthy control group. Overall, the results showed that OAs with hearing or cognitive impairment benefit from AV speech in terms of improved WM performance. In fact, there were a few indications that the AV benefit may be even more robust in these groups than in cognitively healthy OAs. The theoretical and practical implications of these findings and directions for future research are discussed

The Auditory-Visual Speech Benefit on Working Memory in Older Adults with Hearing Impairment

This study examined the effect of auditory-visual (AV) speech stimuli on working memory in hearing impaired participants (HIP) in comparison to age- and education-matched normal elderly controls (NEC). Participants completed a working memory n-back task (0- to 2-back) in which sequences of digits were presented in visual-only (i.e., speech-reading), auditory-only (A-only), and AV conditions. Auditory event-related potentials (ERP) were collected to assess the relationship between perceptual and working memory processing. The behavioural results showed that both groups were faster in the AV condition in comparison to the unisensory conditions. The ERP data showed perceptual facilitation in the AV condition, in the form of reduced amplitudes and latencies of the auditory N1 and/or P1 components, in the HIP group. Furthermore, a working memory ERP component, the P3, peaked earlier for both groups in the AV condition compared to the A-only condition. In general, the HIP group showed a more robust AV benefit; however, the NECs showed a dose-response relationship between perceptual facilitation and working memory improvement, especially for facilitation of processing speed. Two measures, reaction time and P3 amplitude, suggested that the presence of visual speech cues may have helped the HIP to counteract the demanding auditory processing, to the level that no group differences were evident during the AV modality despite lower performance during the A-only condition. Overall, this study provides support for the theory of an integrated perceptual-cognitive system. The practical significance of these findings is also discussed