A systematic review of simulation studies which compare existing statistical methods to account for non-compliance in randomised controlled trials

Introduction Non-compliance is a common challenge for researchers and may reduce the power of an intention-to-treat analysis. Whilst a per protocol approach attempts to deal with this issue, it can result in biased estimates. Several methods to resolve this issue have been identified in previous reviews, but there is limited evidence supporting their use. This review aimed to identify simulation studies which compare such methods, assess the extent to which certain methods have been investigated and determine their performance under various scenarios. Methods A systematic search of several electronic databases including MEDLINE and Scopus was carried out from conception to 30th November 2022. Included papers were published in a peer-reviewed journal, readily available in the English language and focused on comparing relevant methods in a superiority randomised controlled trial under a simulation study. Articles were screened using these criteria and a predetermined extraction form used to identify relevant information. A quality assessment appraised the risk of bias in individual studies. Extracted data was synthesised using tables, figures and a narrative summary. Both screening and data extraction were performed by two independent reviewers with disagreements resolved by consensus. Results Of 2325 papers identified, 267 full texts were screened and 17 studies finally included. Twelve methods were identified across papers. Instrumental variable methods were commonly considered, but many authors found them to be biased in some settings. Non-compliance was generally assumed to be all-or-nothing and only occurring in the intervention group, although some methods considered it as time-varying. Simulation studies commonly varied the level and type of non-compliance and factors such as effect size and strength of confounding. The quality of papers was generally good, although some lacked detail and justification. Therefore, their conclusions were deemed to be less reliable. Conclusions It is common for papers to consider instrumental variable methods but more studies are needed that consider G-methods and compare a wide range of methods in realistic scenarios. It is difficult to make conclusions about the best method to deal with non-compliance due to a limited body of evidence and the difficulty in combining results from independent simulation studies.</p

Incorporation of patient and public involvement in statistical methodology research: a survey assessing current practices and attitudes of researchers

Background Patient and public involvement (PPI) ensures that research is designed and conducted in a manner that is most beneficial to the individuals whom it will impact. It has an undisputed place in applied research and is required by many funding bodies. However, PPI in statistical methodology research is more challenging and work is needed to identify where and how patients and the public can meaningfully input in this area. Methods A descriptive cross-sectional research study was conducted using an online questionnaire, which asked statistical methodologists about themselves and their experience conducting PPI, either to inform a grant application or during a funded statistical methodology project. The survey included both closed-text responses, which were reported using summary statistics, and open-ended questions for which common themes were identified. Results 119 complete responses were recorded. Individuals who completed the survey displayed an even range of ages, career lengths and positions, with the majority working in academia. 40.3% of participants reported undertaking PPI to inform a grant application and the majority reported that the inclusion of PPI was received positively by the funder. Only 21.0% of participants reported undertaking PPI during a methodological project. 31.0% of individuals thought that PPI was “very” or “extremely” relevant to statistical methodology research, with 45.5% responding “somewhat” and 24.4% answering “not at all” or “not very”. Arguments for including PPI were that it can provide the motivation for research and shape the research question. Negative opinions included that it is too technical for the public to understand, so they cannot have a meaningful impact. Conclusions This survey found that the views of statistical methodologists on the inclusion of PPI in their research are varied, with some individuals having particularly strong opinions, both positive and negative. Whilst this is clearly a divisive topic, one commonly identified theme was that many researchers are willing to try and incorporate meaningful PPI into their research but would feel more confident if they had access to resources such as specialised training, guidelines, and case studies. Plain English summary Patient and public involvement (or PPI) means researchers working in partnership with patients and the public in any part of research. It can include helping decide what the research question is, how to pass on results to the public, and telling researchers what areas are most important to patients and the public. Statistical methods are the tools we use to analyse data. Statistical methodology research involves making sure these tools use our healthcare data in the best way. PPI is essential in health research and is becoming more common in statistical methodology research. But it can be hard to know how to include patients and the public in statistical methodology research. It may seem complex and not directly related to patients. This paper describes the results from a survey we did about the experiences of researchers who have carried out PPI for statistical methodology research. We asked them what they think about it, and how it affects their research. We also asked if they feel confident including PPI in their research, and whether they are given enough help. Researchers had different views about PPI for statistical methodology research. Some people thought PPI was very important in their research, but others weren’t sure. Many people said that they would like more help such as training and guidelines to help them do better PPI in the future.</p