Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomized trials

Evidence of unexplained discrepancies between planned and conducted statistical analyses: a review of randomised trial

Public availability and adherence to prespecified statistical analysis approaches was low in published randomized trials

BACKGROUND AND OBJECTIVE: Prespecification of statistical methods in clinical trial protocols and statistical analysis plans can help to deter bias from p-hacking but is only effective if the prespecified approach is made available. STUDY DESIGN AND SETTING: For 100 randomized trials published in 2018 and indexed in PubMed, we evaluated how often a prespecified statistical analysis approach for the trial's primary outcome was publicly available. For each trial with an available prespecified analysis, we compared this with the trial publication to identify whether there were unexplained discrepancies. RESULTS: Only 12 of 100 trials (12%) had a publicly available prespecified analysis approach for their primary outcome; this document was dated before recruitment began for only two trials. Of the 12 trials with an available prespecified analysis approach, 11 (92%) had one or more unexplained discrepancies. Only 4 of 100 trials (4%) stated that the statistician was blinded until the SAP was signed off, and only 10 of 100 (10%) stated the statistician was blinded until the database was locked. CONCLUSION: For most published trials, there is insufficient information available to determine whether the results may be subject to p-hacking. Where information was available, there were often unexplained discrepancies between the prespecified and final analysis methods

Integrable and superintegrable systems associated with multi-sums of products

We construct and study certain Liouville integrable, superintegrable, and non-commutative integrable systems, which are associated with multi-sums of products.Comment: 26 pages, submitted to Proceedings of the royal society

Accounting for centre-effects in multicentre trials with a binary outcome - when, why, and how?

BACKGROUND: It is often desirable to account for centre-effects in the analysis of multicentre randomised trials, however it is unclear which analysis methods are best in trials with a binary outcome. METHODS: We compared the performance of four methods of analysis (fixed-effects models, random-effects models, generalised estimating equations (GEE), and Mantel-Haenszel) using a re-analysis of a previously reported randomised trial (MIST2) and a large simulation study. RESULTS: The re-analysis of MIST2 found that fixed-effects and Mantel-Haenszel led to many patients being dropped from the analysis due to over-stratification (up to 69% dropped for Mantel-Haenszel, and up to 33% dropped for fixed-effects). Conversely, random-effects and GEE included all patients in the analysis, however GEE did not reach convergence. Estimated treatment effects and p-values were highly variable across different analysis methods. The simulation study found that most methods of analysis performed well with a small number of centres. With a large number of centres, fixed-effects led to biased estimates and inflated type I error rates in many situations, and Mantel-Haenszel lost power compared to other analysis methods in some situations. Conversely, both random-effects and GEE gave nominal type I error rates and good power across all scenarios, and were usually as good as or better than either fixed-effects or Mantel-Haenszel. However, this was only true for GEEs with non-robust standard errors (SEs); using a robust ‘sandwich’ estimator led to inflated type I error rates across most scenarios. CONCLUSIONS: With a small number of centres, we recommend the use of fixed-effects, random-effects, or GEE with non-robust SEs. Random-effects and GEE with non-robust SEs should be used with a moderate or large number of centres

Students' conceptual difficulties in hydrodynamics

We describe a study on the conceptual difficulties faced by college students in understanding hydrodynamics of ideal fluids. This study was based on responses obtained in hundreds of written exams complemented with several oral interviews, which were held with first-year engineering and science university students. Their responses allowed us to identify a series of misconceptions unreported in the literature so far. The study findings demonstrate that the most critical difficulties arise from the students' inability to establish a link between the kinematics and dynamics of moving fluids, and from a lack of understanding regarding how different regions of a system interact

On integrability of Hirota-Kimura type discretizations

We give an overview of the integrability of the Hirota-Kimura discretization method applied to algebraically completely integrable (a.c.i.) systems with quadratic vector fields. Along with the description of the basic mechanism of integrability (Hirota-Kimura bases), we provide the reader with a fairly complete list of the currently available results for concrete a.c.i. systems.Comment: 47 pages, some minor change