Sample size in multistakeholder Delphi surveys: at what minimum sample size do replicability of results stabilise?

Background The minimum sample size for multistakeholder Delphi surveys remains understudied. Drawing from three large international multistakeholder Delphi surveys, this study aimed to: 1) investigate the effect of increasing sample size on replicability of results; 2) assess whether the level of replicability of results differed with participant characteristics: e.g., gender, age, profession. Methods We used data from Delphi surveys to develop guidance for improved reporting of healthcare intervention trials: SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extension for surrogate endpoints (n=175, 22 items rated); CONSORT-SPI, extension for social and psychological interventions (n=333, 77 items rated); and core outcome set for burn care (n=553, 88 items rated). Resampling with replacement was used to draw random subsamples from the participant data set in each of the three surveys. For each subsample, the median value of all rated survey items was calculated and compared to the medians from the full participant data set. The median number (and interquartile range) of medians replicated was used to calculate the percentage replicability (and variability). High replicability was defined as ≥80% and moderate as 60% and <80% Results The average median replicability (variability) as a percentage of total number of items rated from the three datasets was 81% (10%) at a sample size of 60. In one of the datasets (CONSORT-SPI), a ≥80% replicability was reached at a sample size of 80. On average, increasing the sample size from 80 to 160 increased the replicability of results by a further 3% and reduced variability by 1%. For subgroup analysis based on participant characteristics (e.g. gender, age, professional role), using resampled samples of 20 to 100 showed that a sample size of 20 to 30 resulted to moderate replicability levels of 64 to 77%. Conclusion We found that a minimum sample size of 60 to 80 participants in multistakeholder Delphi surveys provide a high level of replicability (≥80%) in the results. For Delphi studies limited to individual stakeholder groups (such as researchers, clinicians, patients), a sample size of 20 to 30 per group may be sufficient

Scoping and targeted reviews to support development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: protocol

Introduction Using a surrogate endpoint as a substitute for a primary patient-relevant outcome enables randomised controlled trials (RCTs) to be conducted more efficiently, that is, with shorter time, smaller sample size and lower cost. However, there is currently no consensus-driven guideline for the reporting of RCTs using a surrogate endpoint as a primary outcome; therefore, we seek to develop SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) extensions to improve the design and reporting of these trials. As an initial step, scoping and targeted reviews will identify potential items for inclusion in the extensions and participants to contribute to a Delphi consensus process. Methods and analysis The scoping review will search and include literature reporting on the current understanding, limitations and guidance on using surrogate endpoints in trials. Relevant literature will be identified through: (1) bibliographic databases; (2) grey literature; (3) handsearching of reference lists and (4) solicitation from experts. Data from eligible records will be thematically analysed into potential items for inclusion in extensions. The targeted review will search for RCT reports and protocols published from 2017 to 2021 in six high impact general medical journals. Trial corresponding author contacts will be listed as potential participants for the Delphi exercise. Ethics and dissemination Ethical approval is not required. The reviews will support the development of SPIRIT and CONSORT extensions for reporting surrogate primary endpoints (surrogate endpoint as the primary outcome). The findings will be published in open-access publications. This review has been prospectively registered in the OSF Registration DOI: 10.17605/OSF.IO/WP3QH

Perceptions of diabetes risk and prevention in Nairobi, Kenya: a qualitative and theory of change development study

Background: Type 2 diabetes is increasing in Kenya, especially in urban settings, and prevention interventions based on local evidence and context are urgently needed. Therefore, this study aimed to explore diabetes risk and co-create a diabetes prevention theory of change in two socioeconomically distinct communities to inform future diabetes prevention interventions. Methods: In-depth interviews were conducted with middle-aged residents in two communities in Nairobi (one low-income (n = 15), one middle-income (n = 14)), and thematically analysed. The theory of change for diabetes prevention was informed by analysis of the in-depth interviews and the Behaviour Change Wheel framework, and reviewed by a sub-set (n = 13) of interviewees. Results: The key factors that influenced diabetes preventive practices in both communities included knowledge and skills for diabetes prevention, understanding of the benefits/consequences of (un)healthy lifestyle, social influences (e.g., upbringing, societal perceptions), and environmental contexts (e.g., access to (un)healthy foods and physical activity facilities). The proposed strategies for diabetes prevention included: increasing knowledge and understanding about diabetes risk and preventive measures particularly in the low-income community; supporting lifestyle modification (e.g., upskilling, goal setting, action planning) in both communities; identifying people at high risk of diabetes through screening in both communities; and creating social and physical environments for lifestyle modification (e.g., positive social influences on healthy living, access to healthy foods and physical activity infrastructure) particularly in the low-income community. Residents from both communities agreed that the strategies were broadly feasible for diabetes prevention but proposed the addition of door-to-door campaigns and community theatre for health education. However, residents from the low-income community were concerned about the lack of government prioritisation for implementing population-level interventions, e.g., improving access to healthy foods and physical activity facilities/infrastructure. Conclusion: Diabetes prevention initiatives in Kenya should involve multicomponent interventions for lifestyle modification including increasing education and upskilling at individual level; promoting social and physical environments that support healthy living at population level; and are particularly needed in low-income communities

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

Introduction Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome. Methods and analysis The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases. Ethics and dissemination The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

Introduction Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome. Methods and analysis The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases. Ethics and dissemination The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums

Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate end points in trials: a scoping review

Objective To synthesize the current literature on the use of surrogate end points, including definitions, acceptability, and limitations of surrogate end points and guidance for their design/reporting, into trial reporting items. Study Design and Setting Literature was identified through searching bibliographic databases (until March 1, 2022) and gray literature sources (until May 27, 2022). Data were thematically analyzed into four categories: (1) definitions, (2) acceptability, (3) limitations and challenges, and (4) guidance, and synthesized into reporting guidance items. Results After screening, 90 documents were included: 79% (n = 71) had data on definitions, 77% (n = 69) on acceptability, 72% (n = 65) on limitations and challenges, and 61% (n = 55) on guidance. Data were synthesized into 17 potential trial reporting items: explicit statements on the use of surrogate end point(s) and justification for their use (items 1–6); methodological considerations, including whether sample size calculations were informed by surrogate validity (items 7–9); reporting of results for composite outcomes containing a surrogate end point (item 10); discussion and interpretation of findings (items 11–14); plans for confirmatory studies, collecting data on the surrogate end point and target outcome, and data sharing (items 15–16); and informing trial participants about using surrogate end points (item 17). Conclusion The review identified and synthesized items on the use of surrogate end points in trials; these will inform the development of the Standard Protocol Items: Recommendations for Interventional Trials–SURROGATE and Consolidated Standards of Reporting Trials–SURROGATE extensions

Protocol for the development of SPIRIT and CONSORT extensions for randomised controlled trials with surrogate primary endpoints: SPIRIT-SURROGATE and CONSORT-SURROGATE

Introduction: Randomised controlled trials (RCTs) may use surrogate endpoints as substitutes and predictors of patient-relevant/participant-relevant final outcomes (eg, survival, health-related quality of life). Translation of effects measured on a surrogate endpoint into health benefits for patients/participants is dependent on the validity of the surrogate; hence, more accurate and transparent reporting on surrogate endpoints is needed to limit misleading interpretation of trial findings. However, there is currently no explicit guidance for the reporting of such trials. Therefore, we aim to develop extensions to the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) and CONSORT (Consolidated Standards of Reporting Trials) reporting guidelines to improve the design and completeness of reporting of RCTs and their protocols using a surrogate endpoint as a primary outcome. Methods and analysis: The project will have four phases: phase 1 (literature reviews) to identify candidate reporting items to be rated in a Delphi study; phase 2 (Delphi study) to rate the importance of items identified in phase 1 and receive suggestions for additional items; phase 3 (consensus meeting) to agree on final set of items for inclusion in the extensions and phase 4 (knowledge translation) to engage stakeholders and disseminate the project outputs through various strategies including peer-reviewed publications. Patient and public involvement will be embedded into all project phases. Ethics and dissemination: The study has received ethical approval from the University of Glasgow College of Medical, Veterinary and Life Sciences Ethics Committee (project no: 200210051). The findings will be published in open-access peer-reviewed publications and presented in conferences, meetings and relevant forums

Definitions, acceptability, limitations, and guidance in the use and reporting of surrogate endpoints in trials: a scoping review

Objective: To synthesise the current literature on the use of surrogate endpoints, including definitions, acceptability, and limitations of surrogate endpoints, and guidance for their design/reporting, into trial reporting items. Study Design and Setting: Literature was identified through searching bibliographic databases (until March 1st, 2022) and grey literature sources (until May 27th, 2022). Data were thematically analysed into four categories: (1) definitions, (2) acceptability, (3) limitations and challenges, and (4) guidance, and synthesised into reporting guidance items. Results: After screening, 90 documents were included: 79% (n=71) had data on definitions, 77% (n=69) on acceptability, 72% (n=65) on limitations and challenges, and 61% (n=55) on guidance. Data were synthesised into 17 potential trial reporting items: explicit statements on the use of surrogate endpoint(s) and justification for their use (items 1-6); methodological considerations, including whether sample size calculations were informed by surrogate validity (items 7-9); reporting of results for composite outcomes containing a surrogate endpoint (item 10); discussion and interpretation of findings (items 11-14); plans for confirmatory studies, collecting data on the surrogate endpoint and target outcome, and data sharing (items 15-16); and informing trial participants about using surrogate endpoints (item 17). Conclusion: The review identified and synthesised items on the use of surrogate endpoints in trials; these will inform the development of the SPIRIT-SURROGATE and CONSORT-SURROGATE extensions

Reporting of surrogate endpoints in randomised controlled trial protocols (SPIRIT-Surrogate): extension checklist with explanation and elaboration

Randomised controlled trials often use surrogate endpoints to substitute for a target outcome (an outcome of direct interest and relevance to trial participants, clinicians, and other stakeholders—eg, all cause mortality) to improve efficiency (through shortened duration of follow-up, reduced sample size, and lower research costs), and for ethical or practical reasons. However, their use has a fundamental limitation in terms of uncertainty of the intervention effect on the target outcome and limited information on potential intervention harms. There have been increasing calls for improved reporting of trial protocols that use surrogate endpoints. This report presents the SPIRIT-Surrogate, an extension of the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist, a consensus driven reporting guideline designed for trial protocols using surrogate endpoints as the primary outcome(s). The SPIRIT-Surrogate extension includes nine items modified from the SPIRIT 2013 checklist. The guideline provides examples and explanations for each item. We recommend that all stakeholders (including trial investigators and sponsors, research ethics reviewers, funders, journal editors, and peer reviewers) use this extension in reporting trial protocols that use surrogate endpoints. Its use will allow for improved design of such trials, improved transparency, and interpretation of findings when trials are completed, and ultimately reduced research waste

Reporting of surrogate endpoints in randomised controlled trial reports (CONSORT-Surrogate): extension checklist with explanation and elaboration

Randomised controlled trials commonly use surrogate endpoints to substitute for a target outcome (outcome of direct interest and relevance to trial participants, clinicians, and other stakeholders—eg, all cause mortality) to improve their efficiency (through shorter trial duration, reduced sample size, and thus lower research costs), or for ethical or practical reasons. But reliance on surrogate endpoints can increase the uncertainty of an intervention’s treatment effect and potential failure to provide adequate information on intervention harms, which has led to calls for improved reporting of trials using surrogate endpoints. This report presents a consensus driven reporting guideline for trials using surrogate endpoints as the primary outcomes—the CONSORT (Consolidated Standards of Reporting Trials) extension checklist: CONSORT-Surrogate. The extension includes nine items modified from the CONSORT 2010 checklist and two new items. Examples and explanations for each item are provided. We recommend that all stakeholders (including trial investigators and sponsors, journal editors and peer reviewers, research ethics reviewers, and funders) use this extension in reporting trial reports using surrogate endpoints. Use of this checklist will improve transparency, interpretation, and usefulness of trial findings, and ultimately reduce research waste