t-tests, non-parametric tests, and large studies—a paradox of statistical practice?

Background During the last 30 years, the median sample size of research studies published in high-impact medical journals has increased manyfold, while the use of non-parametric tests has increased at the expense of t-tests. This paper explores this paradoxical practice and illustrates its consequences. Methods A simulation study is used to compare the rejection rates of the Wilcoxon-Mann-Whitney (WMW) test and the two-sample t-test for increasing sample size. Samples are drawn from skewed distributions with equal means and medians but with a small difference in spread. A hypothetical case study is used for illustration and motivation. Results The WMW test produces, on average, smaller p-values than the t-test. This discrepancy increases with increasing sample size, skewness, and difference in spread. For heavily skewed data, the proportion of p<0.05 with the WMW test can be greater than 90% if the standard deviations differ by 10% and the number of observations is 1000 in each group. The high rejection rates of the WMW test should be interpreted as the power to detect that the probability that a random sample from one of the distributions is less than a random sample from the other distribution is greater than 50%. Conclusions Non-parametric tests are most useful for small studies. Using non-parametric tests in large studies may provide answers to the wrong question, thus confusing readers. For studies with a large sample size, t-tests and their corresponding confidence intervals can and should be used even for heavily skewed data

Recommended confidence intervals for two independent binomial proportions

Abstract The relationship between two independent binomial proportions is commonly estimated and presented using the difference between proportions, the number needed to treat, the ratio of proportions or the odds ratio. Several different confidence intervals are available, but they can produce markedly different results. Some of the traditional approaches, such as the Wald interval for the difference between proportions and the Katz log interval for the ratio of proportions, do not perform well unless the sample size is large. Better intervals are available. This article describes and compares approximate and exact confidence intervals that are -with one exception -easy to calculate or available in common software packages. We illustrate the performances of the intervals and make recommendations for both small and moderate-to-large sample sizes

Low level of MAp44, an inhibitor of the lectin complement pathway, and long-term graft and patient survival; a cohort study of 382 kidney recipients

© 2016 The Author(s). Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Background: Higher incidence of malignancy and infectious diseases in kidney transplant recipients is related to immunosuppressive treatment after transplantation and the recipient’s native immune system. The complement system is an essential component of the innate immunity. The aim of the present study was to investigate the association of effector molecules of the lectin complement pathway with graft and patient survival after kidney transplantation. Methods: Two mannan-binding lectin (MBL) associated proteases, MASP-2 and MASP-3 (activators of the lectin pathway) and two MBL-associated proteins, MAp44 and MAp19 (inhibitors of the lectin pathway) were measured at the time of transplantation in 382 patients (≥17 years old) transplanted in 2000–2001. The cohort was followed until December 31, 2014. Data on patient and graft survival were obtained from the Norwegian Renal Registry. Cox proportional hazard regression models were performed for survival analyses. Results: Low MAp44 level (1st versus 2–4 quartile) was significantly associated with overall mortality; HR 1.52, 95 % CI 1.08–2.14, p = 0.017. In the sub analyses in groups below and above median age (51.7 years), low MAp44 as a predictor of overall mortality was statistically significant only in recipients of ≤51.7 years; HR 2.57, 95 % CI 1.42–4.66, p = 0.002. Furthermore, low MAp44 was associated with mortality due to infectious diseases; HR 2.22, 95 % CI 1.11–4.41, p = 0.023. There was no association between MASP-2, MASP-3 or MAp19 levels and patient mortality. No association between any measured biomarkers and death censored graft loss was found. Conclusions: Low MAp44 level at the time of transplantation was associated with increased overall mortality in kidney recipients of median age of 51.7 years or below and with mortality due to infectious diseases in the whole patient cohort after nearly 14-years of follow up after transplantation. No associations between other effector molecules; MASP-2, MASP-3 or MAp19 and recipient mortality were found, as well as no association of any biomarker with death censored graft loss

Parametric methods outperformed non-parametric methods in comparisons of discrete numerical variables

<p>Abstract</p> <p>Background</p> <p>The number of events per individual is a widely reported variable in medical research papers. Such variables are the most common representation of the general variable type called discrete numerical. There is currently no consensus on how to compare and present such variables, and recommendations are lacking. The objective of this paper is to present recommendations for analysis and presentation of results for discrete numerical variables.</p> <p>Methods</p> <p>Two simulation studies were used to investigate the performance of hypothesis tests and confidence interval methods for variables with outcomes {0, 1, 2}, {0, 1, 2, 3}, {0, 1, 2, 3, 4}, and {0, 1, 2, 3, 4, 5}, using the difference between the means as an effect measure.</p> <p>Results</p> <p>The Welch U test (the T test with adjustment for unequal variances) and its associated confidence interval performed well for almost all situations considered. The Brunner-Munzel test also performed well, except for small sample sizes (10 in each group). The ordinary T test, the Wilcoxon-Mann-Whitney test, the percentile bootstrap interval, and the bootstrap-<it>t </it>interval did not perform satisfactorily.</p> <p>Conclusions</p> <p>The difference between the means is an appropriate effect measure for comparing two independent discrete numerical variables that has both lower and upper bounds. To analyze this problem, we encourage more frequent use of parametric hypothesis tests and confidence intervals.</p

Calculating unreported confidence intervals for paired data

<p>Abstract</p> <p>Background</p> <p>Confidence intervals (or associated standard errors) facilitate assessment of the practical importance of the findings of a health study, and their incorporation into a meta-analysis. For paired design studies, these items are often not reported. Since the descriptive statistics for such studies are usually presented in the same way as for unpaired designs, direct computation of the standard error is not possible without additional information.</p> <p>Methods</p> <p>Elementary, well-known relationships between standard errors and <it>p</it>-values were used to develop computation schemes for paired mean difference, risk difference, risk ratio and odds ratio.</p> <p>Results</p> <p>Unreported confidence intervals for large sample paired binary and numeric data can be computed fairly accurately using simple methods provided the <it>p</it>-value is given. In the case of paired binary data, the design based 2 × 2 table can be reconstructed as well.</p> <p>Conclusions</p> <p>Our results will facilitate appropriate interpretation of paired design studies, and their incorporation into meta-analyses.</p

Outcome based subgroup analysis: a neglected concern

A subgroup of clinical trial subjects identified by baseline characteristics is a proper subgroup while a subgroup determined by post randomization events or measures is an improper subgroup. Both types of subgroups are often analyzed in clinical trial papers. Yet, the extensive scrutiny of subgroup analyses has almost exclusively attended to the former. The analysis of improper subgroups thereby not only flourishes in numerous disguised ways but also does so without a corresponding awareness of its pitfalls. Comparisons of the grade of angina in a heart disease trial, for example, usually include only the survivors. This paper highlights some of the distinct ways in which outcome based subgroup analysis occurs, describes the hazards associated with it, and proposes a simple alternative approach to counter its analytic bias. Data from six published trials show that outcome based subgroup analysis, like proper subgroup analysis, may be performed in a post-hoc fashion, overdone, selectively reported, and over interpreted. Six hypothetical trial scenarios illustrate the forms of hidden bias related to it. That bias can, however, be addressed by assigning clinically appropriate scores to the usually excluded subjects and performing an analysis that includes all the randomized subjects. A greater level of awareness about the practice and pitfalls of outcome based subgroup analysis is needed. When required, such an analysis should maintain the integrity of randomization. This issue needs greater practical and methodologic attention than has been accorded to it thus far

Device-measured physical activity, adiposity and mortality: a harmonised meta-analysis of eight prospective cohort studies.

BACKGROUND: The joint associations of total and intensity-specific physical activity with obesity in relation to all-cause mortality risk are unclear. METHODS: We included 34 492 adults (72% women, median age 62.1 years, 2034 deaths during follow-up) in a harmonised meta-analysis of eight population-based prospective cohort studies with mean follow-up ranging from 6.0 to 14.5 years. Standard body mass index categories were cross-classified with sample tertiles of device-measured total, light-to-vigorous and moderate-to-vigorous physical activity and sedentary time. In five cohorts with waist circumference available, high and low waist circumference was combined with tertiles of moderate-to-vigorous physical activity. RESULTS: There was an inverse dose-response relationship between higher levels of total and intensity-specific physical activity and mortality risk in those who were normal weight and overweight. In individuals with obesity, the inverse dose-response relationship was only observed for total physical activity. Similarly, lower levels of sedentary time were associated with lower mortality risk in normal weight and overweight individuals but there was no association between sedentary time and risk of mortality in those who were obese. Compared with the obese-low total physical activity reference, the HRs were 0.59 (95% CI 0.44 to 0.79) for normal weight-high total activity and 0.67 (95% CI 0.48 to 0.94) for obese-high total activity. In contrast, normal weight-low total physical activity was associated with a higher risk of mortality compared with the obese-low total physical activity reference (1.28; 95% CI 0.99 to 1.67). CONCLUSIONS: Higher levels of physical activity were associated with lower risk of mortality irrespective of weight status. Compared with obesity-low physical activity, there was no survival benefit of being normal weight if physical activity levels were low

Joint associations of accelerometer measured physical activity and sedentary time with all-cause mortality: a harmonised meta-analysis in more than 44 000 middle-aged and older individuals

Funder: National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care East MidlandsFunder: National Institute on AgingFunder: Stockholms Läns Landsting; FundRef: http://dx.doi.org/10.13039/501100004348Funder: Norwegian Directorate for Public HealthFunder: Centrum for Idrottsforskning; FundRef: http://dx.doi.org/10.13039/501100005350Funder: The Coca-Cola CompanyObjectives: To examine the joint associations of accelerometer-measured physical activity and sedentary time with all-cause mortality. Methods: We conducted a harmonised meta-analysis including nine prospective cohort studies from four countries. 44 370 men and women were followed for 4.0 to 14.5 years during which 3451 participants died (7.8% mortality rate). Associations between different combinations of moderate-to-vigorous intensity physical activity (MVPA) and sedentary time were analysed at study level using Cox proportional hazards regression analysis and summarised using random effects meta-analysis. Results: Across cohorts, the average time spent sedentary ranged from 8.5 hours/day to 10.5 hours/day and 8 min/day to 35 min/day for MVPA. Compared with the referent group (highest physical activity/lowest sedentary time), the risk of death increased with lower levels of MVPA and greater amounts of sedentary time. Among those in the highest third of MVPA, the risk of death was not statistically different from the referent for those in the middle (16%; 95% CI 0.87% to 1.54%) and highest (40%; 95% CI 0.87% to 2.26%) thirds of sedentary time. Those in the lowest third of MVPA had a greater risk of death in all combinations with sedentary time; 65% (95% CI 1.25% to 2.19%), 65% (95% CI 1.24% to 2.21%) and 263% (95% CI 1.93% to 3.57%), respectively. Conclusion: Higher sedentary time is associated with higher mortality in less active individuals when measured by accelerometry. About 30–40 min of MVPA per day attenuate the association between sedentary time and risk of death, which is lower than previous estimates from self-reported data

Exact and mid-p confidence intervals for the odds ratio

The odds ratio is a frequently used effect measure for two independent binomial proportions. Unfortunately, the confidence intervals that are available for it in Stata and other standard software packages are generally wider than necessary, particularly for small-sample and exact estimation. The performance of the Cornfield exact interval—the only widely available exact interval for the odds ratio—may be improved by incorporating a small modification attributed to Baptista and Pike (1977, Journal of the Royal Statistical Society, Series C 26: 214–220). A further improvement is achieved when the Baptista Pike method is combined with the mid-p approach. In this article, I present the command merci (mid-p and exact odds-ratio confidence intervals) and its immediate version mercii, which calculate the Cornfield exact, Cornfield mid-p, Baptista–Pike exact, and Baptista Pike mid-p confidence intervals for the odds ratio. I compare these intervals with three well-known logit intervals. I strongly recommend the Baptista–Pike mid-p interval