Auditory and visual scene analysis: an overview.

We perceive the world as stable and composed of discrete objects even though auditory and visual inputs are often ambiguous owing to spatial and temporal occluders and changes in the conditions of observation. This raises important questions regarding where and how 'scene analysis' is performed in the brain. Recent advances from both auditory and visual research suggest that the brain does not simply process the incoming scene properties. Rather, top-down processes such as attention, expectations and prior knowledge facilitate scene perception. Thus, scene analysis is linked not only with the extraction of stimulus features and formation and selection of perceptual objects, but also with selective attention, perceptual binding and awareness. This special issue covers novel advances in scene-analysis research obtained using a combination of psychophysics, computational modelling, neuroimaging and neurophysiology, and presents new empirical and theoretical approaches. For integrative understanding of scene analysis beyond and across sensory modalities, we provide a collection of 15 articles that enable comparison and integration of recent findings in auditory and visual scene analysis.This article is part of the themed issue 'Auditory and visual scene analysis'.B.C.J.M. was supported by the Engineering and Physical Sciences Research Council (UK, grant no. RG78536)

Content or status: Frontal and posterior cortical representations of object category and upcoming task goals in working memory

To optimize task sequences, the brain must differentiate between current and prospective goals. We previously showed that currently and prospectively relevant object representations in working memory can be dissociated within object-selective cortex. Based on other recent studies indicating that a range of brain areas may be involved in distinguishing between currently relevant and prospectively relevant information in working memory, here we conducted multivoxel pattern analyses of fMRI activity in additional posterior areas (specifically early visual cortex and the intraparietal sulcus) as well as frontal areas (specifically the frontal eye fields and lateral prefrontal cortex). We assessed whether these areas represent the memory content, the current versus prospective status of the memory, or both. On each trial, participants memorized an object drawn from three different categories. The object was the target for either a first task (currently relevant), a second task (prospectively relevant), or for neither task (irrelevant). The results revealed a division of labor across brain regions: While posterior areas preferentially coded for content (i.e., the category), frontal areas carried information about the current versus prospective relevance status of the memory, irrespective of the category. Intraparietal sulcus revealed both strong category- and status-sensitivity, consistent with its hub function of combining stimulus and priority signals. Furthermore, cross-decoding analyses revealed that while current and prospective representations were similar prior to search, they became dissimilar during search, in posterior as well as frontal areas. The findings provide further evidence for a dissociation between content and control networks in working memory

Subtle eye movement metrics reveal task-relevant representations prior to visual search

Visual search is thought to be guided by an active visual working memory (VWM) representation of the taskrelevant features, referred to as the search template. In three experiments using a probe technique, we investigated which eye movement metrics reveal which search template is activated prior to the search, and distinguish it from future relevant or no longer relevant VWM content. Participants memorized a target color for a subsequent search task, while being instructed to keep central fixation. Before the search display appeared, we briefly presented two task-irrelevant colored probe stimuli to the left and right from fixation, one of which could match the current target template. In all three experiments, participants made both more and larger eye movements towards the probe matching the target color. The bias was predominantly expressed in microsaccades, 100-250 ms after probe onset. Experiment 2 used a retro-cue technique to show that these metrics distinguish between relevant and dropped representations. Finally, Experiment 3 used a sequential task paradigm, and showed that the same metrics also distinguish between current and prospective search templates. Taken together, we show how subtle eye movements track task-relevant representations for selective attention prior to visual search

Emotional stimuli modulate readiness for action: a transcranial magnetic stimulation study

Emotional stimuli may prime the motor system and facilitate action readiness. Direct evidence for this effect has been shown by recent studies using transcranial magnetic stimulation (TMS). When administered over the primary motor cortex involved in responding, TMS pulses elicit motor-evoked potentials (MEPs) in the represented muscles. The amplitudes of these MEPs reflect the state of corticospinal excitability. Here, we investigated the dynamic effects of induced emotions on action readiness, as reflected by corticospinal excitability. Subjects performed a choice task while viewing task-irrelevant emotional and neutral pictures. The pattern of MEP amplitudes showed a typical increase as the TMS pulse was presented closer in time to the imminent response. This dynamic pattern was amplified by both pleasant and unpleasant emotional stimuli, but more so when unpleasant pictures were viewed. These patterns present novel evidence in support of the notion that emotional stimuli modulate action readiness

Emotional facial expressions reduce neural adaptation to face identity

In human social interactions, facial emotional expressions are a crucial source of information. Repeatedly presented information typically leads to an adaptation of neural responses. However, processing seems sustained with emotional facial expressions. Therefore, we tested whether sustained processing of emotional expressions, especially threat-related expressions, would attenuate neural adaptation. Neutral and emotional expressions (happy, mixed and fearful) of same and different identity were presented at 3 Hz. We used electroencephalography to record the evoked steady-state visual potentials (ssVEP) and tested to what extent the ssVEP amplitude adapts to the same when compared with different face identities. We found adaptation to the identity of a neutral face. However, for emotional faces, adaptation was reduced, decreasing linearly with negative valence, with the least adaptation to fearful expressions. This short and straightforward method may prove to be a valuable new tool in the study of emotional processing

GABAA agonist reduces visual awareness: a masking-EEG experiment

Consciousness can be manipulated in many ways. Here, we seek to understand whether two such ways, visual masking and pharmacological intervention, share a common pathway in manipulating visual consciousness. We recorded EEG from human participants who performed a backward-masking task in which they had to detect a masked figure form its background (masking strength was varied across trials). In a within-subject design, participants received dextromethorphan (a N-methyl-d-aspartate receptor antagonist), lorazepam (LZP; a GABA(A) receptor agonist), scopolamine (a muscarine receptor antagonist), or placebo. The behavioral results show that detection rate decreased with increasing masking strength and that of all the drugs, only LZP induced a further decrease in detection rate. Figure-related ERP signals showed three neural events of interest: (1) an early posterior occipital and temporal generator (94-121 msec) that was not influenced by any pharmacological manipulation nor by masking, (2) a later bilateral perioccipital generator (156-211 msec) that was reduced by masking as well as LZP (but not by any other drugs), and (3) a late bilateral occipital temporal generator (293-387 msec) that was mainly affected by masking. Crucially, only the intermediate neural event correlated with detection performance. In combination with previous findings, these results suggest that LZP and masking both reduce visual awareness by means of modulating late activity in the visual cortex but leave early activation intact. These findings provide the first evidence for a common mechanism for these two distinct ways of manipulating consciousness

The subjective experience of object recognition: comparing metacognition for object detection and object categorization

Perceptual decisions seem to be made automatically and almost instantly. Constructing a unitary subjective conscious experience takes more time. For example, when trying to avoid a collision with a car on a foggy road you brake or steer away in a reflex, before realizing you were in a near accident. This subjective aspect of object recognition has been given little attention. We used metacognition (assessed with confidence ratings) to measure subjective experience during object detection and object categorization for degraded and masked objects, while objective performance was matched. Metacognition was equal for degraded and masked objects, but categorization led to higher metacognition than did detection. This effect turned out to be driven by a difference in metacognition for correct rejection trials, which seemed to be caused by an asymmetry of the distractor stimulus: It does not contain object-related information in the detection task, whereas it does contain such information in the categorization task. Strikingly, this asymmetry selectively impacted metacognitive ability when objective performance was matched. This finding reveals a fundamental difference in how humans reflect versus act on information: When matching the amount of information required to perform two tasks at some objective level of accuracy (acting), metacognitive ability (reflecting) is still better in tasks that rely on positive evidence (categorization) than in tasks that rely more strongly on an absence of evidence (detection)

Symptom- and Prevention-Based Testing of COVID-19 in Nursing Home Residents: A Retrospective Cohort Study

Nursing homes (NH) residents with COVID-19 can either be tested because of presence of core symptoms (S-based) or because of transmission prevention (TP-based). The investigated study sample included all NH residents who underwent SARS-CoV-2 RT-PCR testing between March 16, 2020 and May 31, 2020 (n = 380). Clinical symptoms, temperature, and oxygen saturation were extracted from medical records, 7 days before to 14 days after testing. COVID-19 was confirmed in 81 (21%) residents; 36 (44%) S-based and 45 (56%) TP-based: 45. Cycle threshold (CT) values did not differ between the groups. In the 7 days prior to the test falling (32%), somnolence (25%) and fatigue (21%) occurred in both groups. Two days before the test, we observed a stronger decrease in oxygen saturation and an increase in temperature for the S-based group compared to the T-based group that remained up to 10 days after testing. Residents within the S-based group were 2.5 times more likely to increased mortality within 30 days than residents in the TP-based group (HR, 2.56; 95% 1.3–5.2). Although, 73% of the T-based group did eventually develop core symptoms. Thus, attention to falling and daily measures of temperature and oxygen saturation can contribute to earlier detection

The Disability Paradox?:Trajectories of Well-Being in Older Adults With Functional Decline

Objectives: The ‘disability paradox’ (DP) suggests that most older adults maintain subjective well-being (SWB) despite functional decline. However, this may depend the SWB component: positive affect (PA), negative/depressed affect (NA/DA) or life satisfaction (LS). We assessed trajectories of these components in older adults with substantial functional decline. Methods: Data originated from the Longitudinal Aging Study Amsterdam (N = 2545) observed during 1992–2008. Using latent class growth analysis, we distinguished a group with substantial functional decline and examined their SWB trajectories and individual characteristics. Results: The DP occurred more frequently for DA (Men:73%, Women:77%) and LS (Men:14%, Women:83%) than for PA (Men:26%, Women:17%). Higher perceived control (mastery) emerged as the most consistent factor associated with higher odds of the DP. Discussion: We provide a nuanced view of the DP, shifting the question from whether it exists to for which dimension of SWB and for whom it is more or less apparent

Can the meaning of multiple words be integrated unconsciously?

What are the limits of unconscious language processing? Can language circuits process simple grammatical constructions unconsciously and integrate the meaning of several unseen words? Using behavioural priming and electroencephalography (EEG), we studied a specific rule-based linguistic operation traditionally thought to require conscious cognitive control: the negation of valence. In a masked priming paradigm, two masked words were successively (Experiment 1) or simultaneously presented (Experiment 2), a modifier ('not'/'very') and an adjective (e.g. 'good'/'bad'), followed by a visible target noun (e.g. 'peace'/'murder'). Subjects indicated whether the target noun had a positive or negative valence. The combination of these three words could either be contextually consistent (e.g. 'very bad - murder') or inconsistent (e.g. 'not bad - murder'). EEG recordings revealed that grammatical negations could unfold partly unconsciously, as reflected in similar occipito-parietal N400 effects for conscious and unconscious three-word sequences forming inconsistent combinations. However, only conscious word sequences elicited P600 effects, later in time. Overall, these results suggest that multiple unconscious words can be rapidly integrated and that an unconscious negation can automatically 'flip the sign' of an unconscious adjective. These findings not only extend the limits of subliminal combinatorial language processes, but also highlight how consciousness modulates the grammatical integration of multiple words