Predicting Fixations From Deep and Low-Level Features

Learning what properties of an image are associated with human gaze placement is important both for understanding how biological systems explore the environment and for computer vision applications. Recent advances in deep learning for the first time enable us to explain a significant portion of the information expressed in the spatial fixation structure. Our saliency model DeepGaze II uses the VGG network (trained on object recognition in the ImageNet challenge) to convert an image into a high-dimensional feature space which is then readout by a second very simple network to yield a density prediction. DeepGaze II is right now the best performing model for predicting fixations when freeviewing still images (MIT Saliency Benchmark, AUC and sAUC). By retraining on other datasets, we can explore how the features driving fixations change over different tasks or over presentation time. Additionally, the modular architecture of DeepGaze II allows us to quantify how predictive certain features are for fixations. We demonstrate this by replacing the VGG network with very simple isotropic mean-luminance-contrast features and end up with a network that outperforms all previous saliency models before the models that used pretrained deep networks (including models with high-level features like Judd or eDN). Using DeepGaze and the Mean-Luminance-Contrast model (MLC), we can separate how much low-level and high-level features contribute to fixation selection in different situations

Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps

Semantic information is important in eye movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the scene attract more fixations than consistent objects. One interpretation of this effect is that fixations are driven toward inconsistent objects because they are semantically more informative. We tested this explanation using contextualized meaning maps, a method that is based on crowd-sourced ratings to quantify the spatial distribution of context-sensitive “meaning” in images. In Experiment 1, we compared gaze data and contextualized meaning maps for images, in which objects-scene consistency was manipulated. Observers fixated more on inconsistent versus consistent objects. However, contextualized meaning maps did not assign higher meaning to image regions that contained semantic inconsistencies. In Experiment 2, a large number of raters evaluated image-regions, which were deliberately selected for their content and expected meaningfulness. The results suggest that the same scene locations were experienced as slightly less meaningful when they contained inconsistent compared to consistent objects. In summary, we demonstrated that — in the context of our rating task — semantically inconsistent objects are experienced as less meaningful than their consistent counterparts and that contextualized meaning maps do not capture prototypical influences of image meaning on gaze guidance

Semantic object-scene inconsistencies affect eye movements, but not in the way predicted by contextualized meaning maps

Semantic information is important in eye movement control. An important semantic influence on gaze guidance relates to object-scene relationships: objects that are semantically inconsistent with the scene attract more fixations than consistent objects. One interpretation of this effect is that fixations are driven toward inconsistent objects because they are semantically more informative. We tested this explanation using contextualized meaning maps, a method that is based on crowd-sourced ratings to quantify the spatial distribution of context-sensitive “meaning” in images. In Experiment 1, we compared gaze data and contextualized meaning maps for images, in which objects-scene consistency was manipulated. Observers fixated more on inconsistent versus consistent objects. However, contextualized meaning maps did not assign higher meaning to image regions that contained semantic inconsistencies. In Experiment 2, a large number of raters evaluated image-regions, which were deliberately selected for their content and expected meaningfulness. The results suggest that the same scene locations were experienced as slightly less meaningful when they contained inconsistent compared to consistent objects. In summary, we demonstrated that — in the context of our rating task — semantically inconsistent objects are experienced as less meaningful than their consistent counterparts and that contextualized meaning maps do not capture prototypical influences of image meaning on gaze guidance

Meaning maps and saliency models based on deep convolutional neural networks are insensitive to image meaning when predicting human fixations

Eye movements are vital for human vision, and it is therefore important to understand how observers decide where to look. Meaning maps (MMs), a technique to capture the distribution of semantic information across an image, have recently been proposed to support the hypothesis that meaning rather than image features guides human gaze. MMs have the potential to be an important tool far beyond eye-movements research. Here, we examine central assumptions underlying MMs. First, we compared the performance of MMs in predicting fixations to saliency models, showing that DeepGaze II – a deep neural network trained to predict fixations based on high-level features rather than meaning – outperforms MMs. Second, we show that whereas human observers respond to changes in meaning induced by manipulating object-context relationships, MMs and DeepGaze II do not. Together, these findings challenge central assumptions underlying the use of MMs to measure the distribution of meaning in images

A novel formulation of inhaled sodium cromoglicate (PA101) in idiopathic pulmonary fibrosis and chronic cough: a randomised, double-blind, proof-of-concept, phase 2 trial

Background Cough can be a debilitating symptom of idiopathic pulmonary fibrosis (IPF) and is difficult to treat. PA101 is a novel formulation of sodium cromoglicate delivered via a high-efficiency eFlow nebuliser that achieves significantly higher drug deposition in the lung compared with the existing formulations. We aimed to test the efficacy and safety of inhaled PA101 in patients with IPF and chronic cough and, to explore the antitussive mechanism of PA101, patients with chronic idiopathic cough (CIC) were also studied. Methods This pilot, proof-of-concept study consisted of a randomised, double-blind, placebo-controlled trial in patients with IPF and chronic cough and a parallel study of similar design in patients with CIC. Participants with IPF and chronic cough recruited from seven centres in the UK and the Netherlands were randomly assigned (1:1, using a computer-generated randomisation schedule) by site staff to receive PA101 (40 mg) or matching placebo three times a day via oral inhalation for 2 weeks, followed by a 2 week washout, and then crossed over to the other arm. Study participants, investigators, study staff, and the sponsor were masked to group assignment until all participants had completed the study. The primary efficacy endpoint was change from baseline in objective daytime cough frequency (from 24 h acoustic recording, Leicester Cough Monitor). The primary efficacy analysis included all participants who received at least one dose of study drug and had at least one post-baseline efficacy measurement. Safety analysis included all those who took at least one dose of study drug. In the second cohort, participants with CIC were randomly assigned in a study across four centres with similar design and endpoints. The study was registered with ClinicalTrials.gov (NCT02412020) and the EU Clinical Trials Register (EudraCT Number 2014-004025-40) and both cohorts are closed to new participants. Findings Between Feb 13, 2015, and Feb 2, 2016, 24 participants with IPF were randomly assigned to treatment groups. 28 participants with CIC were enrolled during the same period and 27 received study treatment. In patients with IPF, PA101 reduced daytime cough frequency by 31·1% at day 14 compared with placebo; daytime cough frequency decreased from a mean 55 (SD 55) coughs per h at baseline to 39 (29) coughs per h at day 14 following treatment with PA101, versus 51 (37) coughs per h at baseline to 52 (40) cough per h following placebo treatment (ratio of least-squares [LS] means 0·67, 95% CI 0·48–0·94, p=0·0241). By contrast, no treatment benefit for PA101 was observed in the CIC cohort; mean reduction of daytime cough frequency at day 14 for PA101 adjusted for placebo was 6·2% (ratio of LS means 1·27, 0·78–2·06, p=0·31). PA101 was well tolerated in both cohorts. The incidence of adverse events was similar between PA101 and placebo treatments, most adverse events were mild in severity, and no severe adverse events or serious adverse events were reported. Interpretation This study suggests that the mechanism of cough in IPF might be disease specific. Inhaled PA101 could be a treatment option for chronic cough in patients with IPF and warrants further investigation

Risk profiles and one-year outcomes of patients with newly diagnosed atrial fibrillation in India: Insights from the GARFIELD-AF Registry.

BACKGROUND: The Global Anticoagulant Registry in the FIELD-Atrial Fibrillation (GARFIELD-AF) is an ongoing prospective noninterventional registry, which is providing important information on the baseline characteristics, treatment patterns, and 1-year outcomes in patients with newly diagnosed non-valvular atrial fibrillation (NVAF). This report describes data from Indian patients recruited in this registry. METHODS AND RESULTS: A total of 52,014 patients with newly diagnosed AF were enrolled globally; of these, 1388 patients were recruited from 26 sites within India (2012-2016). In India, the mean age was 65.8 years at diagnosis of NVAF. Hypertension was the most prevalent risk factor for AF, present in 68.5% of patients from India and in 76.3% of patients globally (P < 0.001). Diabetes and coronary artery disease (CAD) were prevalent in 36.2% and 28.1% of patients as compared with global prevalence of 22.2% and 21.6%, respectively (P < 0.001 for both). Antiplatelet therapy was the most common antithrombotic treatment in India. With increasing stroke risk, however, patients were more likely to receive oral anticoagulant therapy [mainly vitamin K antagonist (VKA)], but average international normalized ratio (INR) was lower among Indian patients [median INR value 1.6 (interquartile range {IQR}: 1.3-2.3) versus 2.3 (IQR 1.8-2.8) (P < 0.001)]. Compared with other countries, patients from India had markedly higher rates of all-cause mortality [7.68 per 100 person-years (95% confidence interval 6.32-9.35) vs 4.34 (4.16-4.53), P < 0.0001], while rates of stroke/systemic embolism and major bleeding were lower after 1 year of follow-up. CONCLUSION: Compared to previously published registries from India, the GARFIELD-AF registry describes clinical profiles and outcomes in Indian patients with AF of a different etiology. The registry data show that compared to the rest of the world, Indian AF patients are younger in age and have more diabetes and CAD. Patients with a higher stroke risk are more likely to receive anticoagulation therapy with VKA but are underdosed compared with the global average in the GARFIELD-AF. CLINICAL TRIAL REGISTRATION-URL: http://www.clinicaltrials.gov. Unique identifier: NCT01090362

Multiorgan MRI findings after hospitalisation with COVID-19 in the UK (C-MORE): a prospective, multicentre, observational cohort study

Introduction: The multiorgan impact of moderate to severe coronavirus infections in the post-acute phase is still poorly understood. We aimed to evaluate the excess burden of multiorgan abnormalities after hospitalisation with COVID-19, evaluate their determinants, and explore associations with patient-related outcome measures. Methods: In a prospective, UK-wide, multicentre MRI follow-up study (C-MORE), adults (aged ≥18 years) discharged from hospital following COVID-19 who were included in Tier 2 of the Post-hospitalisation COVID-19 study (PHOSP-COVID) and contemporary controls with no evidence of previous COVID-19 (SARS-CoV-2 nucleocapsid antibody negative) underwent multiorgan MRI (lungs, heart, brain, liver, and kidneys) with quantitative and qualitative assessment of images and clinical adjudication when relevant. Individuals with end-stage renal failure or contraindications to MRI were excluded. Participants also underwent detailed recording of symptoms, and physiological and biochemical tests. The primary outcome was the excess burden of multiorgan abnormalities (two or more organs) relative to controls, with further adjustments for potential confounders. The C-MORE study is ongoing and is registered with ClinicalTrials.gov, NCT04510025. Findings: Of 2710 participants in Tier 2 of PHOSP-COVID, 531 were recruited across 13 UK-wide C-MORE sites. After exclusions, 259 C-MORE patients (mean age 57 years [SD 12]; 158 [61%] male and 101 [39%] female) who were discharged from hospital with PCR-confirmed or clinically diagnosed COVID-19 between March 1, 2020, and Nov 1, 2021, and 52 non-COVID-19 controls from the community (mean age 49 years [SD 14]; 30 [58%] male and 22 [42%] female) were included in the analysis. Patients were assessed at a median of 5·0 months (IQR 4·2–6·3) after hospital discharge. Compared with non-COVID-19 controls, patients were older, living with more obesity, and had more comorbidities. Multiorgan abnormalities on MRI were more frequent in patients than in controls (157 [61%] of 259 vs 14 [27%] of 52; p&lt;0·0001) and independently associated with COVID-19 status (odds ratio [OR] 2·9 [95% CI 1·5–5·8]; padjusted=0·0023) after adjusting for relevant confounders. Compared with controls, patients were more likely to have MRI evidence of lung abnormalities (p=0·0001; parenchymal abnormalities), brain abnormalities (p&lt;0·0001; more white matter hyperintensities and regional brain volume reduction), and kidney abnormalities (p=0·014; lower medullary T1 and loss of corticomedullary differentiation), whereas cardiac and liver MRI abnormalities were similar between patients and controls. Patients with multiorgan abnormalities were older (difference in mean age 7 years [95% CI 4–10]; mean age of 59·8 years [SD 11·7] with multiorgan abnormalities vs mean age of 52·8 years [11·9] without multiorgan abnormalities; p&lt;0·0001), more likely to have three or more comorbidities (OR 2·47 [1·32–4·82]; padjusted=0·0059), and more likely to have a more severe acute infection (acute CRP &gt;5mg/L, OR 3·55 [1·23–11·88]; padjusted=0·025) than those without multiorgan abnormalities. Presence of lung MRI abnormalities was associated with a two-fold higher risk of chest tightness, and multiorgan MRI abnormalities were associated with severe and very severe persistent physical and mental health impairment (PHOSP-COVID symptom clusters) after hospitalisation. Interpretation: After hospitalisation for COVID-19, people are at risk of multiorgan abnormalities in the medium term. Our findings emphasise the need for proactive multidisciplinary care pathways, with the potential for imaging to guide surveillance frequency and therapeutic stratification

Post-acute COVID-19 neuropsychiatric symptoms are not associated with ongoing nervous system injury

A proportion of patients infected with severe acute respiratory syndrome coronavirus 2 experience a range of neuropsychiatric symptoms months after infection, including cognitive deficits, depression and anxiety. The mechanisms underpinning such symptoms remain elusive. Recent research has demonstrated that nervous system injury can occur during COVID-19. Whether ongoing neural injury in the months after COVID-19 accounts for the ongoing or emergent neuropsychiatric symptoms is unclear. Within a large prospective cohort study of adult survivors who were hospitalized for severe acute respiratory syndrome coronavirus 2 infection, we analysed plasma markers of nervous system injury and astrocytic activation, measured 6 months post-infection: neurofilament light, glial fibrillary acidic protein and total tau protein. We assessed whether these markers were associated with the severity of the acute COVID-19 illness and with post-acute neuropsychiatric symptoms (as measured by the Patient Health Questionnaire for depression, the General Anxiety Disorder assessment for anxiety, the Montreal Cognitive Assessment for objective cognitive deficit and the cognitive items of the Patient Symptom Questionnaire for subjective cognitive deficit) at 6 months and 1 year post-hospital discharge from COVID-19. No robust associations were found between markers of nervous system injury and severity of acute COVID-19 (except for an association of small effect size between duration of admission and neurofilament light) nor with post-acute neuropsychiatric symptoms. These results suggest that ongoing neuropsychiatric symptoms are not due to ongoing neural injury