Calculating the sample size required for developing a clinical prediction model.

Clinical prediction models aim to predict outcomes in individuals, to inform diagnosis or prognosis in healthcare. Hundreds of prediction models are published in the medical literature each year, yet many are developed using a dataset that is too small for the total number of participants or outcome events. This leads to inaccurate predictions and consequently incorrect healthcare decisions for some individuals. In this article, the authors provide guidance on how to calculate the sample size required to develop a clinical prediction model

Exposure-Response Estimates for Diesel Engine Exhaust and Lung Cancer Mortality Based on Data from Three Occupational Cohorts

Background: Diesel engine exhaust (DEE) has recently been classified as a known human carcinogen. Objective: We derived a meta-exposure–response curve (ERC) for DEE and lung cancer mortality and estimated lifetime excess risks (ELRs) of lung cancer mortality based on assumed occupational and environmental exposure scenarios. Methods: We conducted a meta-regression of lung cancer mortality and cumulative exposure to elemental carbon (EC), a proxy measure of DEE, based on relative risk (RR) estimates reported by three large occupational cohort studies (including two studies of workers in the trucking industry and one study of miners). Based on the derived risk function, we calculated ELRs for several lifetime occupational and environmental exposure scenarios and also calculated the fractions of annual lung cancer deaths attributable to DEE. Results: We estimated a lnRR of 0.00098 (95% CI: 0.00055, 0.0014) for lung cancer mortality with each 1-μg/m3-year increase in cumulative EC based on a linear meta-regression model. Corresponding lnRRs for the individual studies ranged from 0.00061 to 0.0012. Estimated numbers of excess lung cancer deaths through 80 years of age for lifetime occupational exposures of 1, 10, and 25 μg/m3 EC were 17, 200, and 689 per 10,000, respectively. For lifetime environmental exposure to 0.8 μg/m3 EC, we estimated 21 excess lung cancer deaths per 10,000. Based on broad assumptions regarding past occupational and environmental exposures, we estimated that approximately 6% of annual lung cancer deaths may be due to DEE exposure. Conclusions: Combined data from three U.S. occupational cohort studies suggest that DEE at levels common in the workplace and in outdoor air appear to pose substantial excess lifetime risks of lung cancer, above the usually acceptable limits in the United States and Europe, which are generally set at 1/1,000 and 1/100,000 based on lifetime exposure for the occupational and general population, respectively. Citation: Vermeulen R, Silverman DT, Garshick E, Vlaanderen J, Portengen L, Steenland K. 2014. Exposure-response estimates for diesel engine exhaust and lung cancer mortality based on data from three occupational cohorts. Environ Health Perspect 122:172–177; http://dx.doi.org/10.1289/ehp.130688

Emergency open cholecystectomy is associated with markedly lower incidence of postoperative nausea and vomiting (PONV) than elective open cholecystectomy: a retrospective cohort study

<p>Abstract</p> <p>Background</p> <p>During a previous study to define and compare incidence risks of postoperative nausea and vomiting (PONV) for elective laparoscopic and open cholecystectomy at two hospitals in Jamaica, secondary analysis comparing PONV risk in elective open cholecystectomy to that after emergency open cholecystectomy suggested that it was markedly reduced in the latter group. The decision was made to collect data on an adequate sample of emergency open cholecystectomy cases and further explore this unexpected finding in a separate study.</p> <p>Methods</p> <p>Data were collected for 91 emergency open cholecystomy cases identified at the two paricipating hospitals from May 2007 retrograde, as was done for the 175 elective open cholecystectomy cases (from the aforementioned study) with which the emergency cases were to be compared. Variables selected for extraction and statistical analysis included all those known, suspected and plausibly associated with the risk of PONV and with urgency of surgery.</p> <p>Results</p> <p>Emergency open cholecystectomy was associated with a markedly reduced incidence risk of PONV compared to elective open cholecystectomy (6.6% versus 28.6%, P < 0.001). The suppressive effect of emergency increased after adjustment for confounders in a multivariable logistic regression model (odds ratio 0.103, P < 0.001). This finding also identifies, by extrapolation, an association between reduced risk of PONV and preoperative nausea and vomiting, which occurred in 80.2% of emergency cases in the 72 hour period preceding surgery.</p> <p>Conclusions</p> <p>The incidence risk of postoperative nausea and vomiting is markedly decreased after emergency open cholecystectomy compared to elective open cholecystectomy. The study, by extrapolation, also identifies a paradoxical association between pre-operative nausea and vomiting, observed in 80.2% of emergency cases, and suppression of PONV. This association, if confirmed in prospective cohort studies, may have implications for PONV prophylaxis if it can be exploited at a sub-clinical level.</p

A tutorial on individualized treatment effect prediction from randomized trials with a binary endpoint.

Randomized trials typically estimate average relative treatment effects, but decisions on the benefit of a treatment are possibly better informed by more individualized predictions of the absolute treatment effect. In case of a binary outcome, these predictions of absolute individualized treatment effect require knowledge of the individual's risk without treatment and incorporation of a possibly differential treatment effect (ie, varying with patient characteristics). In this article, we lay out the causal structure of individualized treatment effect in terms of potential outcomes and describe the required assumptions that underlie a causal interpretation of its prediction. Subsequently, we describe regression models and model estimation techniques that can be used to move from average to more individualized treatment effect predictions. We focus mainly on logistic regression-based methods that are both well-known and naturally provide the required probabilistic estimates. We incorporate key components from both causal inference and prediction research to arrive at individualized treatment effect predictions. While the separate components are well known, their successful amalgamation is very much an ongoing field of research. We cut the problem down to its essentials in the setting of a randomized trial, discuss the importance of a clear definition of the estimand of interest, provide insight into the required assumptions, and give guidance with respect to modeling and estimation options. Simulated data illustrate the potential of different modeling options across scenarios that vary both average treatment effect and treatment effect heterogeneity. Two applied examples illustrate individualized treatment effect prediction in randomized trial data

Minimum sample size for external validation of a clinical prediction model with a binary outcome

In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.</p

Variable selection under multiple imputation using the bootstrap in a prognostic study

Background: Missing data is a challenging problem in many prognostic studies. Multiple imputation (MI) accounts for imputation uncertainty that allows for adequate statistical testing. We developed and tested a methodology combining MI with bootstrapping techniques for studying prognostic variable selection. Method: In our prospective cohort study we merged data from three different randomized controlled trials (RCTs) to assess prognostic variables for chronicity of low back pain. Among the outcome and prognostic variables data were missing in the range of 0 and 48.1%. We used four methods to investigate the influence of respectively sampling and imputation variation: MI only, bootstrap only, and two methods that combine MI and bootstrapping. Variables were selected based on the inclusion frequency of each prognostic variable, i.e. the proportion of times that the variable appeared in the model. The discriminative and calibrative abilities of prognostic models developed by the four methods were assessed at different inclusion levels. Results: We found that the effect of imputation variation on the inclusion frequency was larger than the effect of sampling variation. When MI and bootstrapping were combined at the range of 0% (full model) to 90% of variable selection, bootstrap corrected c-index values of 0.70 to 0.71 and slope values of 0.64 to 0.86 were found. Conclusion: We recommend to account for both imputation and sampling variation in sets of missing data. The new procedure of combining MI with bootstrapping for variable selection, results in multivariable prognostic models with good performance and is therefore attractive to apply on data sets with missing values

A prognostic tool to identify adolescents at high risk of becoming daily smokers

<p>Abstract</p> <p>Background</p> <p>The American Academy of Pediatrics advocates that pediatricians should be involved in tobacco counseling and has developed guidelines for counseling. We present a prognostic tool for use by health care practitioners in both clinical and non-clinical settings, to identify adolescents at risk of becoming daily smokers.</p> <p>Methods</p> <p>Data were drawn from the Nicotine Dependence in Teens (NDIT) Study, a prospective investigation of 1293 adolescents, initially aged 12-13 years, recruited in 10 secondary schools in Montreal, Canada in 1999. Questionnaires were administered every three months for five years. The prognostic tool was developed using estimated coefficients from multivariable logistic models. Model overfitting was corrected using bootstrap cross-validation. Goodness-of-fit and predictive ability of the models were assessed by R², the c-statistic, and the Hosmer-Lemeshow test.</p> <p>Results</p> <p>The 1-year and 2-year probability of initiating daily smoking was a joint function of seven individual characteristics: age; ever smoked; ever felt like you needed a cigarette; parent(s) smoke; sibling(s) smoke; friend(s) smoke; and ever drank alcohol. The models were characterized by reasonably good fit and predictive ability. They were transformed into user-friendly tables such that the risk of daily smoking can be easily computed by summing points for responses to each item. The prognostic tool is also available on-line at <url>http://episerve.chumontreal.qc.ca/calculation_risk/daily-risk/daily_smokingadd.php</url>.</p> <p>Conclusions</p> <p>The prognostic tool to identify youth at high risk of daily smoking may eventually be an important component of a comprehensive tobacco control system.</p

Exploring the association of dual use of the VHA and Medicare with mortality: separating the contributions of inpatient and outpatient services

<p>Abstract</p> <p>Background</p> <p>Older veterans may use both the Veterans Health Administration (VHA) and Medicare, but the association of dual use with health outcomes is unclear. We examined the association of indirect measures of dual use with mortality.</p> <p>Methods</p> <p>Our secondary analysis used survey, claims, and National Death Index data from the Survey on Assets and Health Dynamics among the Oldest Old. The analytic sample included 1,521 men who were Medicare beneficiaries. Veterans were classified as dual users when their self-reported number of hospital episodes or physician visits exceeded that in their Medicare claims. Veterans reporting inpatient or outpatient visits but having no Medicare claims were classified as VHA-only users. Proportional hazards regression was used.</p> <p>Results</p> <p>897 (59%) of the men were veterans, of whom 134 (15%) were dual users. Among dual users, 60 (45%) met the criterion based on inpatient services, 54 (40%) based on outpatient services, and 20 (15%) based on both. 766 men (50%) died. Adjusting for covariates, the independent effect of any dual use was a 38% increased mortality risk (AHR = 1.38; p = .02). Dual use based on outpatient services marginally increased mortality risk by 45% (AHR = 1.45; p = .06), and dual use based on both inpatient and outpatient services increased the risk by 98% (AHR = 1.98; p = .02).</p> <p>Conclusion</p> <p>Indirect measures of dual use were associated with increased mortality risk. New strategies to better coordinate care, such as shared medical records, should be considered.</p

"Predictability of body mass index for diabetes: Affected by the presence of metabolic syndrome?"

<p>Abstract</p> <p>Background</p> <p>Metabolic syndrome (MetS) and body mass index (BMI, kg.m^-2) are established independent risk factors in the development of diabetes; we prospectively examined their relative contributions and joint relationship with incident diabetes in a Middle Eastern cohort.</p> <p>Method</p> <p>participants of the ongoing Tehran lipid and glucose study are followed on a triennial basis. Among non-diabetic participants aged≥ 20 years at baseline (8,121) those with at least one follow-up examination (5,250) were included for the current study. Multivariate logistic regression models were used to estimate sex-specific adjusted odd ratios (ORs) and 95% confidence intervals (CIs) of baseline BMI-MetS categories (normal weight without MetS as reference group) for incident diabetes among 2186 men and 3064 women, aged ≥ 20 years, free of diabetes at baseline.</p> <p>Result</p> <p>During follow up (median 6.5 years); there were 369 incident diabetes (147 in men). In women without MetS, the multivariate adjusted ORs (95% CIs) for overweight (BMI 25-30 kg/m2) and obese (BMI≥30) participants were 2.3 (1.2-4.3) and 2.2 (1.0-4.7), respectively. The corresponding ORs for men without MetS were 1.6 (0.9-2.9) and 3.6 (1.5-8.4) respectively. As compared to the normal-weight/without MetS, normal-weight women and men with MetS, had a multivariate-adjusted ORs for incident diabetes of 8.8 (3.7-21.2) and 3.1 (1.3-7.0), respectively. The corresponding ORs for overweight and obese women with MetS reached to 7.7 (4.0-14.9) and 12.6 (6.9-23.2) and for men reached to 3.4(2.0-5.8) and 5.7(3.9-9.9), respectively.</p> <p>Conclusion</p> <p>This study highlights the importance of screening for MetS in normal weight individuals. Obesity increases diabetes risk in the absence of MetS, underscores the need for more stringent criteria to define healthy metabolic state among obese individuals. Weight reduction measures, thus, should be encouraged in conjunction with achieving metabolic targets not addressed by current definition of MetS, both in every day encounter and public health setting.</p