Temporal recalibration for improving prognostic model development and risk predictions in settings where survival is improving over time.

BACKGROUND:Prognostic models are typically developed in studies covering long time periods. However, if more recent years have seen improvements in survival, then using the full dataset may lead to out-of-date survival predictions. Period analysis addresses this by developing the model in a subset of the data from a recent time window, but results in a reduction of sample size. METHODS:We propose a new approach, called temporal recalibration, to combine the advantages of period analysis and full cohort analysis. This approach develops a model in the entire dataset and then recalibrates the baseline survival using a period analysis sample.The approaches are demonstrated utilizing a prognostic model in colon cancer built using both Cox proportional hazards and flexible parametric survival models with data from 1996-2005 from the Surveillance, Epidemiology, and End Results (SEER) Program database. Comparison of model predictions with observed survival estimates were made for new patients subsequently diagnosed in 2006 and followed-up until 2015. RESULTS:Period analysis and temporal recalibration provided more up-to-date survival predictions that more closely matched observed survival in subsequent data than the standard full cohort models. In addition, temporal recalibration provided more precise estimates of predictor effects. CONCLUSION:Prognostic models are typically developed using a full cohort analysis that can result in out-of-date long-term survival estimates when survival has improved in recent years. Temporal recalibration is a simple method to address this, which can be used when developing and updating prognostic models to ensure survival predictions are more closely calibrated with the observed survival of individuals diagnosed subsequently

Additional file 1 of Sample size requirements are not being considered in studies developing prediction models for binary outcomes: a systematic review

Supplementary Material

Bayesian network meta-analysis methods for combining individual participant data and aggregate data from single arm trials and randomised controlled trials

Background: Increasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome. Methods: We propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset. Results: Without adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates. Conclusions: Our findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail

Case-finding and improving patient outcomes for chronic obstructive pulmonary disease in primary care: the BLISS research programme including cluster RCT

Background: Chronic obstructive pulmonary disease is a major contributor to morbidity, mortality and health service costs but is vastly underdiagnosed. Evidence on screening and how best to approach this is not clear. There are also uncertainties around the natural history (prognosis) of chronic obstructive pulmonary disease and how it impacts on work performance. Objectives: Work package 1: to evaluate alternative methods of screening for undiagnosed chronic obstructive pulmonary disease in primary care, with clinical effectiveness and cost-effectiveness analyses and an economic model of a routine screening programme. Work package 2: to recruit a primary care chronic obstructive pulmonary disease cohort, develop a prognostic model [Birmingham Lung Improvement StudieS (BLISS)] to predict risk of respiratory hospital admissions, validate an existing model to predict mortality risk, address some uncertainties about natural history and explore the potential for a home exercise intervention. Work package 3: to identify which factors are associated with employment, absenteeism, presenteeism (working while unwell) and evaluate the feasibility of offering formal occupational health assessment to improve work performance. Design: Work package 1: a cluster randomised controlled trial with household-level randomised comparison of two alternative case-finding approaches in the intervention arm. Work package 2: cohort study – focus groups. Work package 3: subcohort – feasibility study. Setting: Primary care settings in West Midlands, UK. Participants: Work package 1: 74,818 people who have smoked aged 40–79 years without a previous chronic obstructive pulmonary disease diagnosis from 54 general practices. Work package 2: 741 patients with previously diagnosed chronic obstructive pulmonary disease from 71 practices and participants from the work package 1 randomised controlled trial. Twenty-six patients took part in focus groups. Work package 3: occupational subcohort with 248 patients in paid employment at baseline. Thirty-five patients took part in an occupational health intervention feasibility study. Interventions: Work package 1: targeted case-finding – symptom screening questionnaire, administered opportunistically or additionally by post, followed by diagnostic post-bronchodilator spirometry. The comparator was routine care. Work package 2: twenty-three candidate variables selected from literature and expert reviews. Work package 3: sociodemographic, clinical and occupational characteristics; occupational health assessment and recommendations. Main outcome measures: Work package 1: yield (screen-detected chronic obstructive pulmonary disease) and cost-effectiveness of case-finding; effectiveness of screening on respiratory hospitalisation and mortality after approximately 4 years. Work package 2: respiratory hospitalisation within 2 years, and barriers to and facilitators of physical activity. Work package 3: work performance – feasibility and acceptability of the occupational health intervention and study processes. Results: Work package 1: targeted case-finding resulted in greater yield of previously undiagnosed chronic obstructive pulmonary disease than routine care at 1 year [n = 1278 (4%) vs.n = 337 (1%), respectively; adjusted odds ratio 7.45, 95% confidence interval 4.80 to 11.55], and a model-based estimate of a regular screening programme suggested an incremental cost-effectiveness ratio of £16,596 per additional quality-adjusted life-year gained. However, long-term follow-up of the trial showed that at ≈4 years there was no clear evidence that case-finding, compared with routine practice, was effective in reducing respiratory admissions (adjusted hazard ratio 1.04, 95% confidence interval 0.73 to1.47) or mortality (hazard ratio 1.15, 95% confidence interval 0.82 to 1.61). Work package 2: 2305 patients, comprising 1564 with previously diagnosed chronic obstructive pulmonary disease and 741 work package 1 participants (330 with and 411 without obstruction), were recruited. The BLISS prognostic model among cohort participants with confirmed airflow obstruction (n = 1894) included 6 of 23 candidate variables (i.e. age, Chronic Obstructive Pulmonary Disease Assessment Test score, 12-month respiratory admissions, body mass index, diabetes and forced expiratory volume in 1 second percentage predicted). After internal validation and adjustment (uniform shrinkage factor 0.87, 95% confidence interval 0.72 to 1.02), the model discriminated well in predicting 2-year respiratory hospital admissions (c-statistic 0.75, 95% confidence interval 0.72 to 0.79). In focus groups, physical activity engagement was related to self-efficacy and symptom severity. Work package 3: in the occupational subcohort, increasing dyspnoea and exposure to inhaled irritants were associated with lower work productivity at baseline. Longitudinally, increasing exacerbations and worsening symptoms, but not a decline in airflow obstruction, were associated with absenteeism and presenteeism. The acceptability of the occupational health intervention was low, leading to low uptake and low implementation of recommendations and making a full trial unfeasible. Limitations: Work package 1: even with the most intensive approach, only 38% of patients responded to the case-finding invitation. Management of case-found patients with chronic obstructive pulmonary disease in primary care was generally poor, limiting interpretation of the long-term effectiveness of case-finding on clinical outcomes. Work package 2: the components of the BLISS model may not always be routinely available and calculation of the score requires a computerised system. Work package 3: relatively few cohort participants were in paid employment at baseline, limiting the interpretation of predictors of lower work productivity. Conclusions: This programme has addressed some of the major uncertainties around screening for undiagnosed chronic obstructive pulmonary disease and has resulted in the development of a novel, accurate model for predicting respiratory hospitalisation in people with chronic obstructive pulmonary disease and the inception of a primary care chronic obstructive pulmonary disease cohort for longer-term follow-up. We have also identified factors that may affect work productivity in people with chronic obstructive pulmonary disease as potential targets for future intervention. Future work: We plan to obtain data for longer-term follow-up of trial participants at 10 years. The BLISS model needs to be externally validated. Our primary care chronic obstructive pulmonary disease cohort is a unique resource for addressing further questions to better understand the prognosis of chronic obstructive pulmonary disease. Trial registration: Current Controlled Trials ISRCTN14930255. Funding: This project was funded by the National Institute for Health Research (NIHR) Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 9, No. 13. See the NIHR Journals Library website for further project information.</p