Methods used to assess outcome consistency in clinical studies: A literature-based evaluation.

Evaluation studies of outcomes used in clinical research and their consistency are appearing more frequently in the literature, as a key part of the core outcome set (COS) development. Current guidance suggests such evaluation studies should use systematic review methodology as their default. We aimed to examine the methods used. We searched the Core Outcome Measures in Effectiveness Trials (COMET) database (up to May 2019) supplementing it with additional resources. We included evaluation studies of outcome consistency in clinical studies across health subjects and used a subset of A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 (items 1-9) to assess their methods. Of 93 included evaluation studies of outcome consistency (90 full reports, three summaries), 91% (85/93) reported performing literature searches in at least one bibliographic database, and 79% (73/93) was labelled as a "systematic review". The evaluations varied in terms of satisfying AMSTAR 2 criteria, such that 81/93 (87%) had implemented PICO in the research question, whereas only 5/93 (6%) had included the exclusions list. None of the evaluation studies explained how inconsistency of outcomes was detected, however, 80/90 (88%) concluded inconsistency in individual outcomes (66%, 55/90) or outcome domains (20%, 18/90). Methods used in evaluation studies of outcome consistency in clinical studies differed considerably. Despite frequent being labelled as a "systematic review", adoption of systematic review methodology is selective. While the impact on COS development is unknown, authors of these studies should refrain from labelling them as "systematic review" and focus on ensuring that the methods used to generate the different outcomes and outcome domains are reported transparently

ADRIC: Adverse Drug Reactions In Children - a programme of research using mixed methods

Aims To comprehensively investigate the incidence, nature and risk factors of adverse drug reactions (ADRs) in a hospital-based population of children, with rigorous assessment of causality, severity and avoidability, and to assess the consequent impact on children and families. We aimed to improve the assessment of ADRs by development of new tools to assess causality and avoidability, and to minimise the impact on families by developing better strategies for communication. Review methods Two prospective observational studies, each over 1 year, were conducted to assess ADRs in children associated with admission to hospital, and those occurring in children who were in hospital for longer than 48 hours. We conducted a comprehensive systematic review of ADRs in children. We used the findings from these studies to develop and validate tools to assess causality and avoidability of ADRs, and conducted interviews with parents and children who had experienced ADRs, using these findings to develop a leaflet for parents to inform a communication strategy about ADRs. Results The estimated incidence of ADRs detected in children on admission to hospital was 2.9% [95% confidence interval (CI) 2.5% to 3.3%]. Of the reactions, 22.1% (95% CI 17% to 28%) were either definitely or possibly avoidable. Prescriptions originating in the community accounted for 44 out of 249 (17.7%) of ADRs, the remainder originating from hospital. A total of 120 out of 249 (48.2%) reactions resulted from treatment for malignancies. Off-label and/or unlicensed (OLUL) medicines were more likely to be implicated in an ADR than authorised medicines [relative risk (RR) 1.67, 95% CI 1.38 to 2.02; p  48 hours, the overall incidence of definite and probable ADRs based on all admissions was 15.9% (95% CI 15.0 to 16.8). Opiate analgesic drugs and drugs used in general anaesthesia (GA) accounted for > 50% of all drugs implicated in ADRs. The odds ratio of an OLUL drug being implicated in an ADR compared with an authorised drug was 2.25 (95% CI 1.95 to 2.59; p < 0.001). Risk factors identified were exposure to a GA, age, oncology treatment and number of medicines. The systematic review estimated that the incidence rates for ADRs causing hospital admission ranged from 0.4% to 10.3% of all children [pooled estimate of 2.9% (95% CI 2.6% to 3.1%)] and from 0.6% to 16.8% of all children exposed to a drug during hospital stay. New tools to assess causality and avoidability of ADRs have been developed and validated. Many parents described being dissatisfied with clinician communication about ADRs, whereas parents of children with cancer emphasised confidence in clinician management of ADRs and the way clinicians communicated about medicines. The accounts of children and young people largely reflected parents’ accounts. Clinicians described using all of the features of communication that parents wanted to see, but made active decisions about when and what to communicate to families about suspected ADRs, which meant that communication may not always match families’ needs and expectations. We developed a leaflet to assist clinicians in communicating ADRs to parents. Conclusion The Adverse Drug Reactions In Children (ADRIC) programme has provided the most comprehensive assessment, to date, of the size and nature of ADRs in children presenting to, and cared for in, hospital, and the outputs that have resulted will improve the management and understanding of ADRs in children and adults within the NHS. Recommendations for future research: assess the values that parents and children place on the use of different medicines and the risks that they will find acceptable within these contexts; focusing on high-risk drugs identified in ADRIC, determine the optimum drug dose for children through the development of a gold standard practice for the extrapolation of adult drug doses, alongside targeted pharmacokinetic/pharmacodynamic studies; assess the research and clinical applications of the Liverpool Causality Assessment Tool and the Liverpool Avoidability Assessment Tool; evaluate, in more detail, morbidities associated with anaesthesia and surgery in children, including follow-up in the community and in the home setting and an assessment of the most appropriate treatment regimens to prevent pain, vomiting and other postoperative complications; further evaluate strategies for communication with families, children and young people about ADRs; and quantify ADRs in other settings, for example critical care and neonatology

The COMET Handbook: version 1.0

The selection of appropriate outcomes is crucial when designing clinical trials in order to compare the effects of different interventions directly. For the findings to influence policy and practice, the outcomes need to be relevant and important to key stakeholders including patients and the public, health care professionals and others making decisions about health care. It is now widely acknowledged that insufficient attention has been paid to the choice of outcomes measured in clinical trials. Researchers are increasingly addressing this issue through the development and use of a core outcome set, an agreed standardised collection of outcomes which should be measured and reported, as a minimum, in all trials for a specific clinical area. Accumulating work in this area has identified the need for guidance on the development, implementation, evaluation and updating of core outcome sets. This Handbook, developed by the COMET Initiative, brings together current thinking and methodological research regarding those issues. We recommend a four-step process to develop a core outcome set. The aim is to update the contents of the Handbook as further research is identified

Protocol for the development of a salutogenic intrapartum core outcome set (SIPCOS)

Background: Maternity intrapartum care research and clinical care more often focus on outcomes that minimise or prevent adverse health rather than on what constitutes positive health and wellbeing (salutogenesis). This was highlighted recently in a systematic review of reviews of intrapartum reported outcomes where only 8% of 1648 individual outcomes, from 102 systematic reviews, were agreed as being salutogenically-focused. Added to this is variation in the outcomes measured in individual studies rendering it very difficult for researchers to synthesise, fully, the evidence from studies on a particular topic. One of the suggested ways to address this is to develop and apply an agreed standardised set of outcomes, known as a ‘core outcome set’ (COS). In this paper we present a protocol for the development of a salutogenic intrapartum COS (SIPCOS) for use in maternity care research and a SIPCOS for measuring in daily intrapartum clinical care. Methods: The study proposes three phases in developing the final SIPCOSs. Phase one, which is complete, involved the conduct of a systematic review of reviews to identify a preliminary list of salutogenically-focused outcomes that had previously been reported in systematic reviews of intrapartum interventions. Sixteen unique salutogenically-focused outcome categories were identified. Phase two will involve prioritising these outcomes, from the perspective of key stakeholders (users of maternity services, clinicians and researchers) by asking them to rate the importance of each outcome for inclusion in the SIPCOSs. A final consensus meeting (phase three) will be held, bringing international stakeholders together to review the preliminary SIPCOSs resulting from the survey and to agree and finalise the final SIPCOSs for use in future maternity care research and daily clinical care. Discussion: The expectation in developing the SIPCOSs is that they will be collected and reported in all future studies evaluating intrapartum interventions and measured/recorded in future intrapartum clinical care, as routine, alongside other outcomes also deemed important in the context of the study or clinical scenario. Using the SIPCOSs in this way, will promote and encourage standardised measurements of positive health outcomes in maternity care, into the future

Defining and evaluating novel procedures for involving patients in Core Outcome Set research: creating a meaningful long list of candidate outcome domains

Background Tinnitus is a complex audiological condition affecting many different domains of everyday life. Clinical trials of tinnitus interventions measure and report those outcome domains inconsistently and this hinders direct comparison between study findings. To address this problem, an ongoing project is developing a Core Outcome Set; an agreed list of outcome domains to be measured and reported in all future trials. Part of this project uses a consensus methodology (‘Delphi’ survey), whereby all relevant stakeholders identify important and critical outcome domains from a long list of candidates. This article addresses a gap in the patient involvement literature by describing and reflecting on our involvement of patients to create a meaningful long list of candidate outcome domains. Methods Two Public Research Partners with lived experience of tinnitus reviewed an initial list of 124 outcome domains over two face-to-face workshops. With the Study Management Team, they interpreted each candidate outcome domain and generated a plain language description. Following this, the domain names and descriptions underwent an additional lay review by 14 patients and 5 clinical experts, via an online survey platform. Results Insights gained from the workshops and survey feedback prompted substantial, unforeseen modifications to the long list. These included the reduction of the number of outcome domains (from 124 to 66) via the exclusion of broad concepts and consolidation of equivalent domains or domains outside the scope of the study. Reviewers also applied their lived experience of tinnitus to bring clarity and relevance to domain names and plain language descriptions. Four impacts on the Delphi survey were observed: recruitment exceeded the target by 171%, there were equivalent numbers of patient and professional participants (n=358 and n=312, respectively), feedback was mostly positive, and retention was high (87%). Conclusions Patient involvement was an integral and transformative step of the study design process. Patient involvement was impactful because the online Delphi survey was successful in recruiting and retaining participants, and there were many comments about a positive participatory experience. Seven general methodological features are highlighted which fit with general principles of good patient involvement. These can benefit other Core Outcome Set developers

Core Outcome Set for GROwth restriction: deVeloping Endpoints (COSGROVE).

BACKGROUND: Foetal growth restriction (FGR) refers to a foetus that does not reach its genetically predetermined growth potential. It is well recognised that growth-restricted foetuses are at increased risk of stillbirth, foetal compromise, early neonatal death and neonatal morbidity. Later in life, they are prone to health problems, including increased risk of cardiovascular diseases and neurodevelopmental disorders. Interventions for preventing and treating FGR have been studied in many trials, but evidence is often difficult to synthesise and compare because of differences in the selection and definition of outcomes. To enable future trials to measure similar, meaningful outcomes, we are developing two core outcome sets (COS) - one for prevention and the other for treatment of FGR. METHODS: We will review the literature to identify previously reported outcomes. An international panel of relevant stakeholders who have experience of FGR (parent or carer of a baby that was growth restricted, health professional involved in the care of mothers and babies affected by FGR, a person with expertise in FGR research) will rate the importance of each of those outcomes in a series of three sequential online rounds of a Delphi study. Participants will be able to add items to the proposed list in round 1. A final face-to-face consensus meeting will be held with representatives of each stakeholder group at which a final list of outcomes for inclusion in the COS will be agreed. DISCUSSION: The development of COSs in FGR will ensure the collection and reporting of a minimum dataset agreed by stakeholder consensus and will reduce inconsistencies in the reporting of outcomes across relevant trials. Such standardisation in the reporting of outcomes will improve synthesis of evidence and generalisability of knowledge in the future by reducing heterogeneity in outcomes between trials and thus improve the results of systematic reviews and meta-analyses. Ultimately, we hope that the COSs will lead to an improvement in the quality of evidence-based clinical practice, enhance patient care, and improve the quality and consistency of research. TRIAL REGISTRATION: Not applicable. This study is registered in the Core Outcome Measures for Effectiveness (COMET) database

Development of a core outcome set for orthodontic trials using a mixed-methods approach: Protocol for a multicentre study

© 2017 The Author(s). Background: Orthodontic treatment is commonly undertaken in young people, with over 40% of children in the UK needing treatment and currently one third having treatment, at a cost to the National Health Service in England and Wales of £273 million each year. Most current research about orthodontic care does not consider what patients truly feel about, or want, from treatment, and a diverse range of outcomes is being used with little consistency between studies. This study aims to address these problems, using established methodology to develop a core outcome set for use in future clinical trials of orthodontic interventions in children and young people. Methods/design: This is a mixed-methods study incorporating four distinct stages. The first stage will include a scoping review of the scientific literature to identify primary and secondary outcome measures that have been used in previous orthodontic clinical trials. The second stage will involve qualitative interviews and focus groups with orthodontic patients aged 10 to 16 years to determine what outcomes are important to them. The outcomes elicited from these two stages will inform the third stage of the study in which a long-list of outcomes will be ranked in terms of importance using electronic Delphi surveys involving clinicians and patients. The final stage of the study will involve face-to-face consensus meetings with all stakeholders to discuss and agree on the outcome measures that should be included in the final core outcome set. Discussion: This research will help to inform patients, parents, clinicians and commissioners about outcomes that are important to young people undergoing orthodontic treatment. Adoption of the core outcome set in future clinical trials of orthodontic treatment will make it easier for results to be compared, contrasted and combined. This should translate into improved decision-making by all stakeholders involved. Trial registration: The project has been registered on the Core Outcome Measures in Effectiveness Trials (COMET) website, January 2016