Impact of question order on prioritisation of outcomes in the development of a core outcome set:A randomised controlled trial

Abstract Background Core outcome set (COS) developers increasingly employ Delphi surveys to elicit stakeholders’ opinions of which outcomes to measure and report in trials of a particular condition or intervention. Research outside of Delphi surveys and COS development demonstrates that question order can affect response rates and lead to ‘context effects’, where prior questions determine an item’s meaning and influence responses. This study examined the impact of question order within a Delphi survey for a COS for oesophageal cancer surgery. Methods A randomised controlled trial was nested within the Delphi survey. Patients and health professionals were randomised to receive a survey including clinical and patient-reported outcomes (PROs), where the PRO section appeared first or last. Participants rated (1–9) the importance of 68 items for inclusion in a COS (ratings 7–9 considered ‘essential’). Analyses considered the impact of question order on: (1) survey response rates; (2) participants’ responses; and (3) items retained at end of the survey. Results In total, 116 patients and 71 professionals returned completed surveys. Question order did not affect response rates among patients, but fewer professionals responded when clinical items appeared first (difference = 31.3%, 95% confidence interval [CI] = 13.6–48.9%, P = 0.001). Question order led to different context effects within patients and professionals. While patients rated clinical items highly, irrespective of question order, more PROs were rated essential when appearing last rather than first (difference = 23.7%, 95% CI = 10.5–40.8%). Among professionals, the greatest impact was on clinical items; a higher percentage rated essential when appearing last (difference = 11.6%, 95% CI = 0.0–23.3%). An interaction between question order and the percentage of PRO/clinical items rated essential was observed for patients (P = 0.025) but not professionals (P = 0.357). Items retained for further consideration at the end of the survey were dependent on question order, with discordant items (retained by one question order group only) observed in patients (18/68 [26%]) and professionals (20/68 [29%]). Conclusions In the development of a COS, participants’ ratings of potential outcomes within a Delphi survey depend on the context (order) in which the outcomes are asked, consequently impacting on the final COS. Initial piloting is recommended with consideration of the randomisation of items in the survey to reduce potential bias. Trial registration The randomised controlled trial reported within this paper was nested within the development of a core outcome set to investigate processes in core outcome set development. Outcomes were not health-related and trial registration was not therefore applicable

Multiorgan MRI findings after hospitalisation with COVID-19 in the UK (C-MORE): a prospective, multicentre, observational cohort study

Introduction: The multiorgan impact of moderate to severe coronavirus infections in the post-acute phase is still poorly understood. We aimed to evaluate the excess burden of multiorgan abnormalities after hospitalisation with COVID-19, evaluate their determinants, and explore associations with patient-related outcome measures. Methods: In a prospective, UK-wide, multicentre MRI follow-up study (C-MORE), adults (aged ≥18 years) discharged from hospital following COVID-19 who were included in Tier 2 of the Post-hospitalisation COVID-19 study (PHOSP-COVID) and contemporary controls with no evidence of previous COVID-19 (SARS-CoV-2 nucleocapsid antibody negative) underwent multiorgan MRI (lungs, heart, brain, liver, and kidneys) with quantitative and qualitative assessment of images and clinical adjudication when relevant. Individuals with end-stage renal failure or contraindications to MRI were excluded. Participants also underwent detailed recording of symptoms, and physiological and biochemical tests. The primary outcome was the excess burden of multiorgan abnormalities (two or more organs) relative to controls, with further adjustments for potential confounders. The C-MORE study is ongoing and is registered with ClinicalTrials.gov, NCT04510025. Findings: Of 2710 participants in Tier 2 of PHOSP-COVID, 531 were recruited across 13 UK-wide C-MORE sites. After exclusions, 259 C-MORE patients (mean age 57 years [SD 12]; 158 [61%] male and 101 [39%] female) who were discharged from hospital with PCR-confirmed or clinically diagnosed COVID-19 between March 1, 2020, and Nov 1, 2021, and 52 non-COVID-19 controls from the community (mean age 49 years [SD 14]; 30 [58%] male and 22 [42%] female) were included in the analysis. Patients were assessed at a median of 5·0 months (IQR 4·2–6·3) after hospital discharge. Compared with non-COVID-19 controls, patients were older, living with more obesity, and had more comorbidities. Multiorgan abnormalities on MRI were more frequent in patients than in controls (157 [61%] of 259 vs 14 [27%] of 52; p<0·0001) and independently associated with COVID-19 status (odds ratio [OR] 2·9 [95% CI 1·5–5·8]; padjusted=0·0023) after adjusting for relevant confounders. Compared with controls, patients were more likely to have MRI evidence of lung abnormalities (p=0·0001; parenchymal abnormalities), brain abnormalities (p<0·0001; more white matter hyperintensities and regional brain volume reduction), and kidney abnormalities (p=0·014; lower medullary T1 and loss of corticomedullary differentiation), whereas cardiac and liver MRI abnormalities were similar between patients and controls. Patients with multiorgan abnormalities were older (difference in mean age 7 years [95% CI 4–10]; mean age of 59·8 years [SD 11·7] with multiorgan abnormalities vs mean age of 52·8 years [11·9] without multiorgan abnormalities; p<0·0001), more likely to have three or more comorbidities (OR 2·47 [1·32–4·82]; padjusted=0·0059), and more likely to have a more severe acute infection (acute CRP >5mg/L, OR 3·55 [1·23–11·88]; padjusted=0·025) than those without multiorgan abnormalities. Presence of lung MRI abnormalities was associated with a two-fold higher risk of chest tightness, and multiorgan MRI abnormalities were associated with severe and very severe persistent physical and mental health impairment (PHOSP-COVID symptom clusters) after hospitalisation. Interpretation: After hospitalisation for COVID-19, people are at risk of multiorgan abnormalities in the medium term. Our findings emphasise the need for proactive multidisciplinary care pathways, with the potential for imaging to guide surveillance frequency and therapeutic stratification

Whose picture is this? Children's memory for item and source information

Children aged 3½ to 6½ years viewed pictures of common objects presented either once or three times on one of two consecutive days. A different hand puppet was used to present the pictures on each day, providing both perceptual and temporal cues to source. At test, old (studied) and new (non-studied) pictures were presented for item recognition and source identification. Results showed that both item and source accuracy were higher for older (M = 5; 9 years) than younger children (M = 4; 6 years). Significant interactions between Age and Day of study were found for both item and source accuracy. For younger children, accuracy was higher for pictures studied on Day 1 than Day 2 (significant for source identification but not item recognition), whereas older children showed the opposite pattern: Higher accuracy for Day 2 than Day 1 (significant for item recognition but not source identification). Results are interpreted with respect to proactive interference and response bias. The utility of signal detection theory measures in determining the basis of age differences in performance of source identification is discussed

Safety management training in small aircraft operations: who do you trust?

Small plane pilots and operators are currently being required to upgrade their nontechnical knowledge and skills in order to implement a range of incoming new regulations. Implementation of these changes creates new and challenging issues for small plane operations and operators, including accessing or providing appropriate training and learning opportunities. In addition, there is likely to be an 'interim period' during which there are differences in knowledge, skills and opinions, until the majority of pilots have had a chance to access and integrate new information. Underpinning any current or future nontechnical training programs is the concept of Trust. The question of trust arises in relation to the type and quality of the trainer, the training options available, and also in relation to pilot perceptions of other pilots around them, eg 'which pilots know what' and how 'trustworthy' or reliable they may be in their application of nontechnical information. This paper addresses the issue of pilot trust per se. Results indicate that there is a relatively low level of small plane pilot trust in the nature and usefulness of human factors related training, in other pilots' application of such information, and in increased safety outcomes due to incoming requirements. Pilots indicted higher levels of trust in three preferred training providers (flying schools, club or group, and CASA). The application of trust data in relation to training programs and providers is discussed in an additional paper

Thinking about human memory

Thinking about human memory provides a novel analytical approach to understanding memory that considers the goals of the memory task, the cues and information available, the opportunity to learn, and interference from irrelevant information (noise). Each of the five chapters describing this approach introduces historical ideas and demonstrates how current thinking both differs from and is derived from them. These chapters also contain analyses of current problems designed to demonstrate the power of the approach. In a subsequent chapter, the authors discuss how memory is controlled by the environment, by others, and by ourselves, and then apply their insights to the problem solving of children, our hominin ancestors, and scrub jays. Finally, the questions of how to define episodic memory and how to investigate phylogenetic and developmental changes in memory are addressed

Developmental differences in judgements of recency and frequency: quantitative or qualitative?

The basis of young children's performance of judgments of recency and frequency was investigated using a modified version of Huppert and Piercy's [Huppert, F.A., & Piercy, M. (1978). The role of trace strength in recency and frequency judgements by amnesic and control subjects. Quarterly Journal of Experimental Psychology, 30, 347–354] paradigm. Children aged 4 and 6 years viewed pictures of nameable objects presented either once or three times on either of two consecutive days. At test, children judged how recently (“today” versus “yesterday”) or frequently (once versus three times) each picture had been presented. Developmental improvements in performance were observed for judgments of recency and frequency. Evidence for responding based on episode-specific information was found for both age groups, showing that young children do not confuse recency of presentation with frequency of presentation. There was no evidence for a qualitative shift from reliance on trace strength to reliance on episode specific information across this age range