Penalized Regression Methods With Modified Cross-Validation and Bootstrap Tuning Produce Better Prediction Models

Risk prediction models fitted using maximum likelihood estimation (MLE) are often overfitted resulting in predictions that are too extreme and a calibration slope (CS) less than 1. Penalized methods, such as Ridge and Lasso, have been suggested as a solution to this problem as they tend to shrink regression coefficients toward zero, resulting in predictions closer to the average. The amount of shrinkage is regulated by a tuning parameter, (Formula presented.) commonly selected via cross-validation (“standard tuning”). Though penalized methods have been found to improve calibration on average, they often over-shrink and exhibit large variability in the selected (Formula presented.) and hence the CS. This is a problem, particularly for small sample sizes, but also when using sample sizes recommended to control overfitting. We consider whether these problems are partly due to selecting (Formula presented.) using cross-validation with “training” datasets of reduced size compared to the original development sample, resulting in an over-estimation of (Formula presented.) and, hence, excessive shrinkage. We propose a modified cross-validation tuning method (“modified tuning”), which estimates (Formula presented.) from a pseudo-development dataset obtained via bootstrapping from the original dataset, albeit of larger size, such that the resulting cross-validation training datasets are of the same size as the original dataset. Modified tuning can be easily implemented in standard software and is closely related to bootstrap selection of the tuning parameter (“bootstrap tuning”). We evaluated modified and bootstrap tuning for Ridge and Lasso in simulated and real data using recommended sample sizes, and sizes slightly lower and higher. They substantially improved the selection of (Formula presented.), resulting in improved CS compared to the standard tuning method. They also improved predictions compared to MLE

The use of basic fibroblast growth factor to improve vocal function: A systematic review and meta-analysis

OBJECTIVES: This systematic review and meta-analysis examines if intralaryngeal injection of basic fibroblast growth factor 2 (FGF2) can improve voice outcomes in those with vocal disability. DESIGN: A Systematic review of original human studies reporting voice outcomes following intra-laryngeal injection of basic fibroblast growth factor 2 in those with vocal dysfunction. Databases searched were Medline (1946-July 2022), Embase (1947-July 2022), Cochrane database and Google Scholar. SETTING: Secondary or tertiary care centres that undertook the management of voice pathology Hospital. PARTICIPANTS: Inclusion criteria were original human studies reporting voice outcome measurements following intralaryngeal injection of FGF2 to treat vocal fold atrophy, vocal fold scarring, vocal fold sulcus or vocal fold palsy. Articles not written in English, studies that did not include human subjects and studies where voice outcome measures were not recorded before and after FGF2 injection were excluded from the review. MAIN OUTCOME MEASURES: The primary outcome measure was maximum phonation time. Secondary outcome measures included acoustic analysis, glottic closure, mucosal wave formation, voice handicap index and GRBAS scale. RESULTS: Fourteen articles were included out of a search of 1023 and one article was included from scanning reference lists. All studies had a single arm design without control groups. Conditions treated were vocal fold atrophy (n = 186), vocal cord paralysis (n = 74), vocal fold fibrosis (n = 74) and vocal fold sulcus (n = 56). A meta-analysis of six articles reporting on the use of FGF2 in patients with vocal fold atrophy showed a significant increase of mean maximum phonation time of 5.2 s (95% CI: 3.4-7.0) at 3-6 months following injection. A significant improvement in maximum phonation time, voice handicap index and glottic closure was found following injection in most studies assessed. No major adverse events were reported following injection. CONCLUSIONS: To date, intralaryngeal injection of basic FGF2 appears to be safe and it may be able to improve voice outcomes in those with vocal dysfunction, especially vocal fold atrophy. Randomised controlled trials are needed to further evaluate efficacy and support the wider use of this therapy

A note on obtaining correct marginal predictions from a random intercepts model for binary outcomes.

BACKGROUND: Clustered data with binary outcomes are often analysed using random intercepts models or generalised estimating equations (GEE) resulting in cluster-specific or 'population-average' inference, respectively. METHODS: When a random effects model is fitted to clustered data, predictions may be produced for a member of an existing cluster by using estimates of the fixed effects (regression coefficients) and the random effect for the cluster (conditional risk calculation), or for a member of a new cluster (marginal risk calculation). We focus on the second. Marginal risk calculation from a random effects model is obtained by integrating over the distribution of random effects. However, in practice marginal risks are often obtained, incorrectly, using only estimates of the fixed effects (i.e. by effectively setting the random effects to zero). We compare these two approaches to marginal risk calculation in terms of model calibration. RESULTS: In simulation studies, it has been seen that use of the incorrect marginal risk calculation from random effects models results in poorly calibrated overall marginal predictions (calibration slope <1 and calibration in the large ≠ 0) with mis-calibration becoming worse with higher degrees of clustering. We clarify that this was due to the incorrect calculation of marginal predictions from a random intercepts model and explain intuitively why this approach is incorrect. We show via simulation that the correct calculation of marginal risks from a random intercepts model results in predictions with excellent calibration. CONCLUSION: The logistic random intercepts model can be used to obtain valid marginal predictions by integrating over the distribution of random effects

The Value of Blood-Based Measures of Liver Function and Urate in Lung Cancer Risk Prediction: A Cohort Study and Health Economic Analysis

BACKGROUND: Several studies have reported associations between low-cost blood-based measurements and lung cancer but their role in risk prediction is unclear. We examined the value of expanding lung cancer risk models for targeting low-dose computed tomography (LDCT) to include blood measurements of liver function and urate. METHODS: We analysed a cohort of 388,199 UK Biobank participants with 1,873 events and calculated the c-index and fraction of new information (FNI) for models expanded to include combinations of blood measurements, lung function (forced expiratory volume in 1 second - FEV1), alcohol status and waist circumference. We calculated the hypothetical cost per lung cancer case detected by LDCT for different scenarios using a threshold of ≥ 1.51% risk at 6 years. RESULTS: The c-index was 0.805 (95%CI:0.794-0.816) for the model containing conventional predictors. Expanding to include blood measurements increased the c-index to 0.815 (95%CI: 0.804-0.826;p<0.0001;FNI:0.06). Expanding to include FEV1, alcohol status, and waist circumference increased the c-index to 0.811 (95%CI:0.800-0.822;p<0.0001;FNI:0.04). The c-index for the fully expanded model containing all variables was 0.819 (95%CI:0.808-0.830; p<0.0001;FNI:0.09). Model expansion had a greater impact on the c-index and FNI for people with a history of smoking cigarettes relative to the full cohort. Compared with the conventional risk model, the expanded models reduced the number of participants meeting the criteria for LDCT screening by 15-21%, and lung cancer cases detected by 7-8%. The additional cost per lung cancer case detected relative to the conventional model was £1,018 for the addition of blood tests and £9,775 for the fully expanded model. CONCLUSION: Blood measurements of liver function and urate improved lung cancer risk prediction compared with a model containing conventional risk factors. However, there was no evidence that model expansion would improve the cost per lung cancer case detected in UK health care settings

Relationships between intracranial arterial dolichoectasia and small vessel disease in patients with ischaemic stroke: a systematic review and meta-analysis

BACKGROUND: Intracranial arterial dolichoectasia (IADE) is a common arterial finding of dilation, elongation, or both, affecting large intracranial vessels, and associated with vascular risk factors, including hypertension. Associations of IADE with neuroimaging cerebral small vessel disease (CSVD) may be relevant for diagnosis and prognosis in patients with stroke. The study aimed to conduct an updated systematic review and meta-analysis of observational studies to investigate the relationships of IADE with well-defined CSVD markers in patients with ischaemic stroke. METHODS: We systematically searched PubMed, Embase, and Scopus for studies on IADE in ischaemic stroke patients with fulfilling predefined inclusion criteria. We pooled data to conduct a meta-analysis to compare the prevalence of SVD markers between patients with and without IADE groups using risk ratios (RRs) and 95% confidence intervals (CIs). RESULTS: From 157 retrieved abstracts, we included six studies from seven publications comprising 6102 patients with ischaemic stroke. The mean age of patients was 52.8 years, and 3691 (60.5%) were male. IADE was diagnosed in 11.4% (95% CI 8.9-13.9) (761) of included patients; 51.8% (3160) had hypertension. Compared to patients without IADE, individuals diagnosed with IADE had a significantly increased prevalence of lacune (RR 1.67, 95% CI 1.36-2.06, P < 0.01, I2 = 0.00%), cerebral microbleeds (CMBs) (RR 2.56, 95% CI 1.53-4.28, P < 0.01, I2 = 84.95%) and white matter hyperintensities (WMHs) (RR 2.17, 95% CI 1.84-2.56, P < 0.01, I2 = 0.00%). CONCLUSIONS: In patients with ischaemic stroke, IADE is associated with a higher prevalence of CSVD markers, including lacunes, CMBs, and WMHs. Further studies are needed to clarify the mechanisms underlying these associations and their potential relevance for the understanding, diagnosis, and treatment of CSVD

The value of blood-based measures of liver function and urate in lung cancer risk prediction: A cohort study and health economic analysis

BACKGROUND: Several studies have reported associations between low-cost blood-based measurements and lung cancer but their role in risk prediction is unclear. We examined the value of expanding lung cancer risk models for targeting low-dose computed tomography (LDCT), including blood measurements of liver function and urate. METHODS: We analysed a cohort of 388,199 UK Biobank participants with 1873 events and calculated the c-index and fraction of new information (FNI) for models expanded to include combinations of blood measurements, lung function (forced expiratory volume in 1 s - FEV1), alcohol status and waist circumference. We calculated the hypothetical cost per lung cancer case detected by LDCT for different scenarios using a threshold of ≥ 1.51 % risk at 6 years. RESULTS: The c-index was 0.805 (95 %CI:0.794-0.816) for the model containing conventional predictors. Expanding to include blood measurements increased the c-index to 0.815 (95 %CI: 0.804-0.826;p < 0.0001;FNI:0.06). Expanding to include FEV1, alcohol status, and waist circumference increased the c-index to 0.811 (95 %CI: 0.800-0.822;p < 0.0001;FNI: 0.04). The c-index for the fully expanded model containing all variables was 0.819 (95 %CI:0.808-0.830;p < 0.0001;FNI:0.09). Model expansion had a greater impact on the c-index and FNI for people with a history of smoking cigarettes relative to the full cohort. Compared with the conventional risk model, the expanded models reduced the number of participants meeting the criteria for LDCT screening by 15-21 %, and lung cancer cases detected by 7-8 %. The additional cost per lung cancer case detected relative to the conventional model was £ 1018 for adding blood tests and £ 9775 for the fully expanded model. CONCLUSION: Blood measurements of liver function and urate made a modest improvement to lung cancer risk prediction compared with a model containing conventional risk factors. There was no evidence that model expansion would improve the cost per lung cancer case detected in UK healthcare settings