Dynamics of trimming the content of face representations for categorization in the brain

To understand visual cognition, it is imperative to determine when, how and with what information the human brain categorizes the visual input. Visual categorization consistently involves at least an early and a late stage: the occipito-temporal N170 event related potential related to stimulus encoding and the parietal P300 involved in perceptual decisions. Here we sought to understand how the brain globally transforms its representations of face categories from their early encoding to the later decision stage over the 400 ms time window encompassing the N170 and P300 brain events. We applied classification image techniques to the behavioral and electroencephalographic data of three observers who categorized seven facial expressions of emotion and report two main findings: (1) Over the 400 ms time course, processing of facial features initially spreads bilaterally across the left and right occipito-temporal regions to dynamically converge onto the centro-parietal region; (2) Concurrently, information processing gradually shifts from encoding common face features across all spatial scales (e.g. the eyes) to representing only the finer scales of the diagnostic features that are richer in useful information for behavior (e.g. the wide opened eyes in 'fear'; the detailed mouth in 'happy'). Our findings suggest that the brain refines its diagnostic representations of visual categories over the first 400 ms of processing by trimming a thorough encoding of features over the N170, to leave only the detailed information important for perceptual decisions over the P300

The role of configurality in the Thatcher illusion: an ERP study.

The Thatcher illusion (Thompson in Perception, 9, 483-484, 1980) is often explained as resulting from recognising a distortion of configural information when 'Thatcherised' faces are upright but not when inverted. However, recent behavioural studies suggest that there is an absence of perceptual configurality in upright Thatcherised faces (Donnelly et al. in Attention, Perception & Psychophysics, 74, 1475-1487, 2012) and both perceptual and decisional sources of configurality in behavioural tasks with Thatcherised stimuli (Mestry, Menneer et al. in Frontiers in Psychology, 3, 456, 2012). To examine sources linked to the behavioural experience of the illusion, we studied inversion and Thatcherisation of faces (comparing across conditions in which no features, the eyes, the mouth, or both features were Thatcherised) on a set of event-related potential (ERP) components. Effects of inversion were found at the N170, P2 and P3b. Effects of eye condition were restricted to the N170 generated in the right hemisphere. Critically, an interaction of orientation and eye Thatcherisation was found for the P3b amplitude. Results from an individual with acquired prosopagnosia who can discriminate Thatcherised from typical faces but cannot categorise them or perceive the illusion (Mestry, Donnelly et al. in Neuropsychologia, 50, 3410-3418, 2012) only differed from typical participants at the P3b component. Findings suggest the P3b links most directly to the experience of the illusion. Overall, the study showed evidence consistent with both perceptual and decisional sources and the need to consider both in relation to configurality

Common cortical responses evoked by appearance, disappearance and change of the human face

<p>Abstract</p> <p>Background</p> <p>To segregate luminance-related, face-related and non-specific components involved in spatio-temporal dynamics of cortical activations to a face stimulus, we recorded cortical responses to face appearance (Onset), disappearance (Offset), and change (Change) using magnetoencephalography.</p> <p>Results</p> <p>Activity in and around the primary visual cortex (V1/V2) showed luminance-dependent behavior. Any of the three events evoked activity in the middle occipital gyrus (MOG) at 150 ms and temporo-parietal junction (TPJ) at 250 ms after the onset of each event. Onset and Change activated the fusiform gyrus (FG), while Offset did not. This FG activation showed a triphasic waveform, consistent with results of intracranial recordings in humans.</p> <p>Conclusion</p> <p>Analysis employed in this study successfully segregated four different elements involved in the spatio-temporal dynamics of cortical activations in response to a face stimulus. The results show the responses of MOG and TPJ to be associated with non-specific processes, such as the detection of abrupt changes or exogenous attention. Activity in FG corresponds to a face-specific response recorded by intracranial studies, and that in V1/V2 is related to a change in luminance.</p

Top-down and bottom-up modulation in processing bimodal face/voice stimuli

<p>Abstract</p> <p>Background</p> <p>Processing of multimodal information is a critical capacity of the human brain, with classic studies showing bimodal stimulation either facilitating or interfering in perceptual processing. Comparing activity to congruent and incongruent bimodal stimuli can reveal sensory dominance in particular cognitive tasks.</p> <p>Results</p> <p>We investigated audiovisual interactions driven by stimulus properties (bottom-up influences) or by task (top-down influences) on congruent and incongruent simultaneously presented faces and voices while ERPs were recorded. Subjects performed gender categorisation, directing attention either to faces or to voices and also judged whether the face/voice stimuli were congruent in terms of gender. Behaviourally, the unattended modality affected processing in the attended modality: the disruption was greater for attended voices. ERPs revealed top-down modulations of early brain processing (30-100 ms) over unisensory cortices. No effects were found on N170 or VPP, but from 180-230 ms larger right frontal activity was seen for incongruent than congruent stimuli.</p> <p>Conclusions</p> <p>Our data demonstrates that in a gender categorisation task the processing of faces dominate over the processing of voices. Brain activity showed different modulation by top-down and bottom-up information. Top-down influences modulated early brain activity whereas bottom-up interactions occurred relatively late.</p

From upright to upside-down presentation: A spatio-temporal ERP study of the parametric effect of rotation on face and house processing

<p>Abstract</p> <p>Background</p> <p>While there is a general agreement that picture-plane inversion is more detrimental to face processing than to other seemingly complex visual objects, the origin of this effect is still largely debatable. Here, we address the question of whether face inversion reflects a quantitative or a qualitative change in processing mode by investigating the pattern of event-related potential (ERP) response changes with picture plane rotation of face and house pictures. Thorough analyses of topographical (Scalp Current Density maps, SCD) and dipole source modeling were also conducted.</p> <p>Results</p> <p>We find that whilst stimulus orientation affected in a similar fashion participants' response latencies to make face and house decisions, only the ERPs in the N170 latency range were modulated by picture plane rotation of faces. The pattern of N170 amplitude and latency enhancement to misrotated faces displayed a curvilinear shape with an almost linear increase for rotations from 0° to 90° and a dip at 112.5° up to 180° rotations. A similar discontinuity function was also described for SCD occipito-temporal and temporal current foci with no topographic distribution changes, suggesting that upright and misrotated faces activated similar brain sources. This was confirmed by dipole source analyses showing the involvement of bilateral sources in the fusiform and middle occipital gyri, the activity of which was differentially affected by face rotation.</p> <p>Conclusion</p> <p>Our N170 findings provide support for both the quantitative and qualitative accounts for face rotation effects. Although the qualitative explanation predicted the curvilinear shape of N170 modulations by face misrotations, topographical and source modeling findings suggest that the same brain regions, and thus the same mechanisms, are probably at work when processing upright and rotated faces. Taken collectively, our results indicate that the same processing mechanisms may be involved across the whole range of face orientations, but would operate in a non-linear fashion. Finally, the response tuning of the N170 to rotated faces extends previous reports and further demonstrates that face inversion affects perceptual analyses of faces, which is reflected within the time range of the N170 component.</p

Emotion-Related Visual Mismatch Responses in Schizophrenia: Impairments and Correlations with Emotion Recognition.

BACKGROUND AND OBJECTIVES:Mismatch negativity (MMN) is an event-related potential (ERP) measure of preattentional sensory processing. While deficits in the auditory MMN are robust electrophysiological findings in schizophrenia, little is known about visual mismatch response and its association with social cognitive functions such as emotion recognition in schizophrenia. Our aim was to study the potential deficit in the visual mismatch response to unexpected facial emotions in schizophrenia and its association with emotion recognition impairments, and to localize the sources of the mismatch signals. EXPERIMENTAL DESIGN:The sample comprised 24 patients with schizophrenia and 24 healthy control subjects. Controls were matched individually to patients by gender, age, and education. ERPs were recorded using a high-density 128-channel BioSemi amplifier. Mismatch responses to happy and fearful faces were determined in 2 time windows over six regions of interest (ROIs). Emotion recognition performance and its association with the mismatch response were also investigated. PRINCIPAL OBSERVATIONS:Mismatch signals to both emotional conditions were significantly attenuated in patients compared to controls in central and temporal ROIs. Controls recognized emotions significantly better than patients. The association between overall emotion recognition performance and mismatch response to the happy condition was significant in the 250-360 ms time window in the central ROI. The estimated sources of the mismatch responses for both emotional conditions were localized in frontal regions, where patients showed significantly lower activity. CONCLUSIONS:Impaired generation of mismatch signals indicate insufficient automatic processing of emotions in patients with schizophrenia, which correlates strongly with decreased emotion recognition

Visual Working Memory Load-Related Changes in Neural Activity and Functional Connectivity

BACKGROUND: Visual working memory (VWM) helps us store visual information to prepare for subsequent behavior. The neuronal mechanisms for sustaining coherent visual information and the mechanisms for limited VWM capacity have remained uncharacterized. Although numerous studies have utilized behavioral accuracy, neural activity, and connectivity to explore the mechanism of VWM retention, little is known about the load-related changes in functional connectivity for hemi-field VWM retention. METHODOLOGY/PRINCIPAL FINDINGS: In this study, we recorded electroencephalography (EEG) from 14 normal young adults while they performed a bilateral visual field memory task. Subjects had more rapid and accurate responses to the left visual field (LVF) memory condition. The difference in mean amplitude between the ipsilateral and contralateral event-related potential (ERP) at parietal-occipital electrodes in retention interval period was obtained with six different memory loads. Functional connectivity between 128 scalp regions was measured by EEG phase synchronization in the theta- (4-8 Hz), alpha- (8-12 Hz), beta- (12-32 Hz), and gamma- (32-40 Hz) frequency bands. The resulting matrices were converted to graphs, and mean degree, clustering coefficient and shortest path length was computed as a function of memory load. The results showed that brain networks of theta-, alpha-, beta-, and gamma- frequency bands were load-dependent and visual-field dependent. The networks of theta- and alpha- bands phase synchrony were most predominant in retention period for right visual field (RVF) WM than for LVF WM. Furthermore, only for RVF memory condition, brain network density of theta-band during the retention interval were linked to the delay of behavior reaction time, and the topological property of alpha-band network was negative correlation with behavior accuracy. CONCLUSIONS/SIGNIFICANCE: We suggest that the differences in theta- and alpha- bands between LVF and RVF conditions in functional connectivity and topological properties during retention period may result in the decline of behavioral performance in RVF task

Manipulable Objects Facilitate Cross-Modal Integration in Peripersonal Space

Previous studies have shown that tool use often modifies one's peripersonal space – i.e. the space directly surrounding our body. Given our profound experience with manipulable objects (e.g. a toothbrush, a comb or a teapot) in the present study we hypothesized that the observation of pictures representing manipulable objects would result in a remapping of peripersonal space as well. Subjects were required to report the location of vibrotactile stimuli delivered to the right hand, while ignoring visual distractors superimposed on pictures representing everyday objects. Pictures could represent objects that were of high manipulability (e.g. a cell phone), medium manipulability (e.g. a soap dispenser) and low manipulability (e.g. a computer screen). In the first experiment, when subjects attended to the action associated with the objects, a strong cross-modal congruency effect (CCE) was observed for pictures representing medium and high manipulability objects, reflected in faster reaction times if the vibrotactile stimulus and the visual distractor were in the same location, whereas no CCE was observed for low manipulability objects. This finding was replicated in a second experiment in which subjects attended to the visual properties of the objects. These findings suggest that the observation of manipulable objects facilitates cross-modal integration in peripersonal space

Whisker Movements Reveal Spatial Attention: A Unified Computational Model of Active Sensing Control in the Rat

Spatial attention is most often investigated in the visual modality through measurement of eye movements, with primates, including humans, a widely-studied model. Its study in laboratory rodents, such as mice and rats, requires different techniques, owing to the lack of a visual fovea and the particular ethological relevance of orienting movements of the snout and the whiskers in these animals. In recent years, several reliable relationships have been observed between environmental and behavioural variables and movements of the whiskers, but the function of these responses, as well as how they integrate, remains unclear. Here, we propose a unifying abstract model of whisker movement control that has as its key variable the region of space that is the animal's current focus of attention, and demonstrate, using computer-simulated behavioral experiments, that the model is consistent with a broad range of experimental observations. A core hypothesis is that the rat explicitly decodes the location in space of whisker contacts and that this representation is used to regulate whisker drive signals. This proposition stands in contrast to earlier proposals that the modulation of whisker movement during exploration is mediated primarily by reflex loops. We go on to argue that the superior colliculus is a candidate neural substrate for the siting of a head-centred map guiding whisker movement, in analogy to current models of visual attention. The proposed model has the potential to offer a more complete understanding of whisker control as well as to highlight the potential of the rodent and its whiskers as a tool for the study of mammalian attention