Investigating the Neural Basis of Audiovisual Speech Perception with Intracranial Recordings in Humans

Speech is inherently multisensory, containing auditory information from the voice and visual information from the mouth movements of the talker. Hearing the voice is usually sufficient to understand speech, however in noisy environments or when audition is impaired due to aging or disabilities, seeing mouth movements greatly improves speech perception. Although behavioral studies have well established this perceptual benefit, it is still not clear how the brain processes visual information from mouth movements to improve speech perception. To clarify this issue, I studied the neural activity recorded from the brain surfaces of human subjects using intracranial electrodes, a technique known as electrocorticography (ECoG). First, I studied responses to noisy speech in the auditory cortex, specifically in the superior temporal gyrus (STG). Previous studies identified the anterior parts of the STG as unisensory, responding only to auditory stimulus. On the other hand, posterior parts of the STG are known to be multisensory, responding to both auditory and visual stimuli, which makes it a key region for audiovisual speech perception. I examined how these different parts of the STG respond to clear versus noisy speech. I found that noisy speech decreased the amplitude and increased the across-trial variability of the response in the anterior STG. However, possibly due to its multisensory composition, posterior STG was not as sensitive to auditory noise as the anterior STG and responded similarly to clear and noisy speech. I also found that these two response patterns in the STG were separated by a sharp boundary demarcated by the posterior-most portion of the Heschl’s gyrus. Second, I studied responses to silent speech in the visual cortex. Previous studies demonstrated that visual cortex shows response enhancement when the auditory component of speech is noisy or absent, however it was not clear which regions of the visual cortex specifically show this response enhancement and whether this response enhancement is a result of top-down modulation from a higher region. To test this, I first mapped the receptive fields of different regions in the visual cortex and then measured their responses to visual (silent) and audiovisual speech stimuli. I found that visual regions that have central receptive fields show greater response enhancement to visual speech, possibly because these regions receive more visual information from mouth movements. I found similar response enhancement to visual speech in frontal cortex, specifically in the inferior frontal gyrus, premotor and dorsolateral prefrontal cortices, which have been implicated in speech reading in previous studies. I showed that these frontal regions display strong functional connectivity with visual regions that have central receptive fields during speech perception

Brandright

Trademark law is guilty of overprotection. This overprotection pits both a company’s in-house attorneys against its own marketing professionals and the company itself against its most loyal customers. The result appears illogical, at best, to consumers witnessing the effects of this clash between a company’s marketing needs and perceived legal requirements

Aerospace Medicine and Biology: A continuing bibliography with indexes, supplement 144

This bibliography lists 257 reports, articles, and other documents introduced into the NASA scientific and technical information system in July 1975

The Hierarchy of Stable Distributions and Operators to Trade Off Stability and Performance

Recent work addressing model reliability and generalization has resulted in a variety of methods that seek to proactively address differences between the training and unknown target environments. While most methods achieve this by finding distributions that will be invariant across environments, we will show they do not necessarily find the same distributions which has implications for performance. In this paper we unify existing work on prediction using stable distributions by relating environmental shifts to edges in the graph underlying a prediction problem, and characterize stable distributions as those which effectively remove these edges. We then quantify the effect of edge deletion on performance in the linear case and corroborate the findings in a simulated and real data experiment

Visual, Motor and Attentional Influences on Proprioceptive Contributions to Perception of Hand Path Rectilinearity during Reaching

We examined how proprioceptive contributions to perception of hand path straightness are influenced by visual, motor and attentional sources of performance variability during horizontal planar reaching. Subjects held the handle of a robot that constrained goal-directed movements of the hand to the paths of controlled curvature. Subjects attempted to detect the presence of hand path curvature during both active (subject driven) and passive (robot driven) movements that either required active muscle force production or not. Subjects were less able to discriminate curved from straight paths when actively reaching for a target versus when the robot moved their hand through the same curved paths. This effect was especially evident during robot-driven movements requiring concurrent activation of lengthening but not shortening muscles. Subjects were less likely to report curvature and were more variable in reporting when movements appeared straight in a novel “visual channel” condition previously shown to block adaptive updating of motor commands in response to deviations from a straight-line hand path. Similarly, compromised performance was obtained when subjects simultaneously performed a distracting secondary task (key pressing with the contralateral hand). The effects compounded when these last two treatments were combined. It is concluded that environmental, intrinsic and attentional factors all impact the ability to detect deviations from a rectilinear hand path during goal-directed movement by decreasing proprioceptive contributions to limb state estimation. In contrast, response variability increased only in experimental conditions thought to impose additional attentional demands on the observer. Implications of these results for perception and other sensorimotor behaviors are discussed