Minimum sample size for external validation of a clinical prediction model with a continuous outcome.

Clinical prediction models provide individualized outcome predictions to inform patient counseling and clinical decision making. External validation is the process of examining a prediction model's performance in data independent to that used for model development. Current external validation studies often suffer from small sample sizes, and subsequently imprecise estimates of a model's predictive performance. To address this, we propose how to determine the minimum sample size needed for external validation of a clinical prediction model with a continuous outcome. Four criteria are proposed, that target precise estimates of (i) R2 (the proportion of variance explained), (ii) calibration-in-the-large (agreement between predicted and observed outcome values on average), (iii) calibration slope (agreement between predicted and observed values across the range of predicted values), and (iv) the variance of observed outcome values. Closed-form sample size solutions are derived for each criterion, which require the user to specify anticipated values of the model's performance (in particular R2 ) and the outcome variance in the external validation dataset. A sensible starting point is to base values on those for the model development study, as obtained from the publication or study authors. The largest sample size required to meet all four criteria is the recommended minimum sample size needed in the external validation dataset. The calculations can also be applied to estimate expected precision when an existing dataset with a fixed sample size is available, to help gauge if it is adequate. We illustrate the proposed methods on a case-study predicting fat-free mass in children

A Spatial Analysis of Rift Valley Fever Virus Seropositivity in Domestic Ruminants in Tanzania

Rift Valley fever (RVF) is an acute arthropod-borne viral zoonotic disease primarily occurring in Africa. Since RVF-like disease was reported in Tanzania in 1930, outbreaks of the disease have been reported mainly from the eastern ecosystem of the Great Rift Valley. This cross-sectional study was carried out to describe the variation in RVF virus (RVFV) seropositivity in domestic ruminants between selected villages in the eastern and western Rift Valley ecosystems in Tanzania, and identify potential risk factors. Three study villages were purposively selected from each of the two Rift Valley ecosystems. Serum samples from randomly selected domestic ruminants (n = 1,435) were tested for the presence of specific immunoglobulin G (IgG) and M (IgM), using RVF enzyme-linked immunosorbent assay methods. Mixed effects logistic regression modelling was used to investigate the association between potential risk factors and RVFV seropositivity. The overall RVFV seroprevalence (n = 1,435) in domestic ruminants was 25.8% and species specific seroprevalence was 29.7%, 27.7% and 22.0% in sheep (n = 148), cattle (n = 756) and goats (n = 531), respectively. The odds of seropositivity were significantly higher in animals sampled from the villages in the eastern than those in the western Rift Valley ecosystem (OR = 1.88, CI: 1.41, 2.51; p<0.001), in animals sampled from villages with soils of good than those with soils of poor water holding capacity (OR = 1.97; 95% CI: 1.58, 3.02; p< 0.001), and in animals which had been introduced than in animals born within the herd (OR = 5.08, CI: 2.74, 9.44; p< 0.001). Compared with animals aged 1-2 years, those aged 3 and 4-5 years had 3.40 (CI: 2.49, 4.64; p< 0.001) and 3.31 (CI: 2.27, 4.82, p< 0.001) times the odds of seropositivity. The findings confirm exposure to RVFV in all the study villages, but with a higher prevalence in the study villages from the eastern Rift Valley ecosystem

Minimum sample size calculations for external validation of a clinical prediction model with a time-to-event outcome.

Previous articles in Statistics in Medicine describe how to calculate the sample size required for external validation of prediction models with continuous and binary outcomes. The minimum sample size criteria aim to ensure precise estimation of key measures of a model's predictive performance, including measures of calibration, discrimination, and net benefit. Here, we extend the sample size guidance to prediction models with a time-to-event (survival) outcome, to cover external validation in datasets containing censoring. A simulation-based framework is proposed, which calculates the sample size required to target a particular confidence interval width for the calibration slope measuring the agreement between predicted risks (from the model) and observed risks (derived using pseudo-observations to account for censoring) on the log cumulative hazard scale. Precise estimation of calibration curves, discrimination, and net-benefit can also be checked in this framework. The process requires assumptions about the validation population in terms of the (i) distribution of the model's linear predictor and (ii) event and censoring distributions. Existing information can inform this; in particular, the linear predictor distribution can be approximated using the C-index or Royston's D statistic from the model development article, together with the overall event risk. We demonstrate how the approach can be used to calculate the sample size required to validate a prediction model for recurrent venous thromboembolism. Ideally the sample size should ensure precise calibration across the entire range of predicted risks, but must at least ensure adequate precision in regions important for clinical decision-making. Stata and R code are provided

Minimum sample size for external validation of a clinical prediction model with a binary outcome

In prediction model research, external validation is needed to examine an existing model's performance using data independent to that for model development. Current external validation studies often suffer from small sample sizes and consequently imprecise predictive performance estimates. To address this, we propose how to determine the minimum sample size needed for a new external validation study of a prediction model for a binary outcome. Our calculations aim to precisely estimate calibration (Observed/Expected and calibration slope), discrimination (C-statistic), and clinical utility (net benefit). For each measure, we propose closed-form and iterative solutions for calculating the minimum sample size required. These require specifying: (i) target SEs (confidence interval widths) for each estimate of interest, (ii) the anticipated outcome event proportion in the validation population, (iii) the prediction model's anticipated (mis)calibration and variance of linear predictor values in the validation population, and (iv) potential risk thresholds for clinical decision-making. The calculations can also be used to inform whether the sample size of an existing (already collected) dataset is adequate for external validation. We illustrate our proposal for external validation of a prediction model for mechanical heart valve failure with an expected outcome event proportion of 0.018. Calculations suggest at least 9835 participants (177 events) are required to precisely estimate the calibration and discrimination measures, with this number driven by the calibration slope criterion, which we anticipate will often be the case. Also, 6443 participants (116 events) are required to precisely estimate net benefit at a risk threshold of 8%. Software code is provided.</p

Minimal reporting improvement after peer review in reports of covid-19 prediction models: systematic review.

OBJECTIVE: To assess improvement in the completeness of reporting COVID-19 prediction models after the peer review process. STUDY DESIGN AND SETTING: Studies included in a living systematic review of COVID-19 prediction models, with both pre-print and peer-reviewed published versions available, were assessed. The primary outcome was the change in percentage adherence to the TRIPOD reporting guidelines between pre-print and published manuscripts. RESULTS: 19 studies were identified including seven (37%) model development studies, two external validations of existing models (11%), and 10 (53%) papers reporting on both development and external validation of the same model. Median percentage adherence amongst pre-print versions was 33% (min-max: 10 to 68%). The percentage adherence of TRIPOD components increased from pre-print to publication in 11/19 studies (58%), with adherence unchanged in the remaining eight studies. The median change in adherence was just 3 percentage points (pp, min-max: 0-14pp) across all studies. No association was observed between the change in percentage adherence and pre-print score, journal impact factor, or time between journal submission and acceptance. CONCLUSIONS: Pre-print reporting quality of COVID-19 prediction modelling studies is poor and did not improve much after peer review, suggesting peer review had a trivial effect on the completeness of reporting during the pandemic

A frameshift mutation of the chloroplast matK coding region is associated with chlorophyll deficiency in the Cryptomeria japonica virescent mutant Wogon-Sugi

Wogon-Sugi has been reported as a cytoplasmically inherited virescent mutant selected from a horticultural variety of Cryptomeria japonica. Although previous studies of plastid structure and inheritance indicated that at least some mutations are encoded by the chloroplast genome, the causative gene responsible for the primary chlorophyll deficiency in Wogon-Sugi, has not been identified. In this study, we identified this gene by genomic sequencing of chloroplast DNA and genetic analysis. Chloroplast DNA sequencing of 16 wild-type and 16 Wogon-Sugi plants showed a 19-bp insertional sequence in the matK coding region in the Wogon-Sugi. This insertion disrupted the matK reading frame. Although an indel mutation in the ycf1 and ycf2 coding region was detected in Wogon-Sugi, sequence variations similar to that of Wogon-Sugi were also detected in several wild-type lines, and they maintained the reading frame. Genetic analysis of the 19 bp insertional mutation in the matK coding region showed that it was found only in the chlorophyll-deficient sector of 125 full-sibling seedlings. Therefore, the 19-bp insertion in the matK coding region is the most likely candidate at present for a mutation underlying the Wogon-Sugi phenotype

The Relationship Between Alcohol Symptoms and Consumption Among Older Drinkers

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/66310/1/j.1530-0277.1996.tb01104.x.pd

Predicting the risk of acute kidney injury in primary care: derivation and validation of STRATIFY-AKI

Background: Antihypertensives reduce the risk of cardiovascular disease but are also associated with harms including acute kidney injury (AKI). Few data exist to guide clinical decision making regarding these risks. Aim: To develop a prediction model estimating the risk of AKI in people potentially indicated for antihypertensive treatment. Design and setting: Observational cohort study using routine primary care data from the Clinical Practice Research Datalink (CPRD) in England. Method: People aged ≥40 years, with at least one blood pressure measurement between 130 mmHg and 179 mmHg were included. Outcomes were admission to hospital or death with AKI within 1, 5, and 10 years. The model was derived with data from CPRD GOLD (n = 1 772 618), using a Fine–Gray competing risks approach, with subsequent recalibration using pseudo-values. External validation used data from CPRD Aurum (n = 3 805 322). Results: The mean age of participants was 59.4 years and 52% were female. The final model consisted of 27 predictors and showed good discrimination at 1, 5, and 10 years (C-statistic for 10-year risk 0.821, 95% confidence interval [CI] = 0.818 to 0.823). There was some overprediction at the highest predicted probabilities (ratio of observed to expected event probability for 10-year risk 0.633, 95% CI = 0.621 to 0.645), affecting patients with the highest risk. Most patients (>95%) had a low 1- to 5-year risk of AKI, and at 10 years only 0.1% of the population had a high AKI and low CVD risk. Conclusion: This clinical prediction model enables GPs to accurately identify patients at high risk of AKI, which will aid treatment decisions. As the vast majority of patients were at low risk, such a model may provide useful reassurance that most antihypertensive treatment is safe and appropriate while flagging the few for whom this is not the case

Quantum Gravity in 2+1 Dimensions: The Case of a Closed Universe

In three spacetime dimensions, general relativity drastically simplifies, becoming a ``topological'' theory with no propagating local degrees of freedom. Nevertheless, many of the difficult conceptual problems of quantizing gravity are still present. In this review, I summarize the rather large body of work that has gone towards quantizing (2+1)-dimensional vacuum gravity in the setting of a spatially closed universe.Comment: 61 pages, draft of review for Living Reviews; comments, criticisms, additions, missing references welcome; v2: minor changes, added reference