Predicting outcomes in patients with kidney disease: methodology and clinical applications

In part 1 of the thesis Predicting Outcomes in Patients with Kidney Disease, key differences between etiological and prediction research are explored and it is shown that observational research often conflates etiology and prediction which leads to incorrect causal conclusions. A framework for the external validation of prognostic models is provided and it is shown how competing events can be dealt with when externally validating a time-to-event prognostic model. These results are applicable to many clinical research fields, including nephrology as exemplified in part 2. Within the six applied chapters in part 2, prediction models for various adverse outcomes in patients with advanced kidney disease are identified, validated and developed. The thesis provides a broad overview of prognostic model applications in patients with chronic kidney disease, including comprehensive external validation studies for kidney failure prediction models, mortality prediction models and graft failure prediction models. Models to predict mortality on conservative care and dialysis and models to predict adverse outcomes after kidney transplantation were developed and validated. These results may improve shared decision-making processes and individualized medicine for patients with kidney disease.ChipSoft NierstichtingLUMC / Geneeskund

Con: Most clinical risk scores are useless

While developing prediction models has become quite popular both in nephrology and in medicine in general, most models have not been implemented in clinical practice on a larger scale. This should be no surprise, as the majority of published models has been shown to be poorly reported and often developed using inappropriate methods. The main problems identified relate to either using too few candidate predictors (based on univariable P </p

The association between pain perception and care dependency in older nursing home residents: a prospective cohort study

Objectives: Maintenance of independence is a challenge for nursing home residents whose pain is often substantial. The objective of this study was to explore the relationship between pain perception and care dependency in a population of Dutch nursing home residents.Design: Prospective cohort study. Setting and participants: Dutch nursing home residents aged 65 or older, excluding residents with a severe cognitive impairment.Methods: The Numeric Rating Scale (NRS) was used to rate pain perception from 0 to 10 in half-point increments and the Care Dependency Scale (CDS) to measure care dependency, with scores ranging from 15 (completely care dependent) to 75 (fully independent). Both measurements were repeated after a 2-month follow-up. Multiple linear regression analysis was used to adjust for potential confounders. Missing data were dealt with by performing tenfold multiple imputation.Results: A total of 1256 residents (65% women, mean age 83 years) were included. At baseline, the median NRS pain score was 3.0 (interquartile range 0.0-6.0) and the mean CDS score was 55.9 (SD 11.5). Cross-sectionally, for 1-point increase in pain score, care dependency increased 0.65 points [95% confidence interval (CI) 0.46-0.83]. More pain at baseline was associated with slightly lower care dependency after 2 months (beta 0.20, 95% CI 0.01-0.39). Compared with residents whose pain decreased over 2 months, residents with stable pain or increased pain had a 2.27-point (95% CI 0.83-3.70) and 2.39 point (95% CI 0.87-3.90) greater increase in care dependency, respectively.Conclusions and implications: Pain perception and care dependency are associated in a population of older nursing home residents, and stable or increased pain is associated with increased care dependency progression. The findings of this study emphasize that pain and care dependency should not be assessed nor treated independently. (C) 2020 The Author(s). Published by Elsevier Inc. on behalf of AMDA - The Society for Post-Acute and Long-Term Care Medicine. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).Clinical epidemiolog

Towards the best kidney failure prediction tool: a systematic review and selection aid

Background. Prediction tools that identify chronic kidney disease (CKD) patients at a high risk of developing kidney failure have the potential for great clinical value, but limited uptake. The aim of the current study is to systematically review all available models predicting kidney failure in CKD patients, organize empirical evidence on their validity and ultimately provide guidance in the interpretation and uptake of these tools.Methods. PubMed and EMBASE were searched for relevant articles. Titles, abstracts and full-text articles were sequentially screened for inclusion by two independent researchers. Data on study design, model development and performance were extracted. The risk of bias and clinical usefulness were accessed and combined in order to provide recommendations on which models to use.Results. Of 2183 screened studies, a total of 42 studies were included in the current review. Most studies showed high discriminatory capacity and the included predictors had large overlap. Overall, the risk of bias was high. Slightly less than half the studies (48%) presented enough detail for the use of their prediction tool in practice and few models were externally validated.Conclusions. The current systematic review may be used as a tool to select the most appropriate and robust prognostic model for various settings. Although some models showed great potential, many lacked clinical relevance due to being developed in a prevalent patient population with a wide range of disease severity. Future research efforts should focus on external validation and impact assessment in clinically relevant patient populations.Clinical epidemiolog

Appraising prediction research: a guide and meta-review on bias and applicability assessment using the Prediction model Risk Of Bias ASsessment Tool (PROBAST)

Over the past few years, a large number of prediction models have been published, often of poor methodological quality. Seemingly objective and straightforward, prediction models provide a risk estimate for the outcome of interest, usually based on readily available clinical information. Yet, using models of substandard methodological rigour, especially without external validation, may result in incorrect risk estimates and consequently misclassification. To assess and combat bias in prediction research the prediction model risk of bias assessment tool (PROBAST) was published in 2019. This risk of bias (ROB) tool includes four domains and 20 signalling questions highlighting methodological flaws, and provides guidance in assessing the applicability of the model. In this paper, the PROBAST will be discussed, along with an in-depth review of two commonly encountered pitfalls in prediction modelling that may induce bias: overfitting and composite endpoints. We illustrate the prevalence of potential bias in prediction models with a meta-review of 50 systematic reviews that used the PROBAST to appraise their included studies, thus including 1510 different studies on 2104 prediction models. All domains showed an unclear or high ROB; these results were markedly stable over time, highlighting the urgent need for attention on bias in prediction research. This article aims to do just that by providing (1) the clinician with tools to evaluate the (methodological) quality of a clinical prediction model, (2) the researcher working on a review with methods to appraise the included models, and (3) the researcher developing a model with suggestions to improve model quality.Clinical epidemiolog

Predicting mortality risk on dialysis and conservative care: development and internal validation of a prediction tool for older patients with advanced chronic kidney disease

Background. Conservative care (CC) may be a valid alternative to dialysis for certain older patients with advanced chronic kidney disease (CKD). A model that predicts patient prognosis on both treatment pathways could be of value in shared decision-making. Therefore, the aim is to develop a prediction tool that predicts the mortality risk for the same patient for both dialysis and CC from the time of treatment decision.Methods. CKD Stage 4/5 patients aged >= 70 years, treated at a single centre in the Netherlands, were included between 2004 and 2016. Predictors were collected at treatment decision and selected based on literature and an expert panel. Outcome was 2-year mortality. Basic and extended logistic regression models were developed for both the dialysis and CC groups. These models were internally validated with bootstrapping. Model performance was assessed with discrimination and calibration.Results. In total, 366 patients were included, of which 126 chose CC. Pre-selected predictors for the basic model were age, estimated glomerular filtration rate, malignancy and cardiovascular disease. Discrimination was moderate, with optimism-corrected C-statistics ranging from 0.675 to 0.750. Calibration plots showed good calibration.Conclusions. A prediction tool that predicts 2-year mortality was developed to provide older advanced CKD patients with individualized prognosis estimates for both dialysis and CC. Future studies are needed to test whether our findings hold in other CKD populations. Following external validation, this prediction tool could be used to compare a patient's prognosis on both dialysis and CC, and help to inform treatment decision-making.Clinical epidemiolog

Development and external validation study combining existing models and recent data into an up-to-date prediction model for evaluating kidneys from older deceased donors for transplantation

With a rising demand for kidney transplantation, reliable pre-transplant assessment of organ quality becomes top priority. In clinical practice, physicians are regularly in doubt whether suboptimal kidney offers from older donors should be accepted. Here, we externally validate existing prediction models in a European population of older deceased donors, and subsequently developed and externally validated an adverse outcome prediction tool. Recipients of kidney grafts from deceased donors 50 years of age and older were included from the Netherlands Organ Transplant Registry (NOTR) and United States organ transplant registry from 2006-2018. The predicted adverse outcome was a composite of graft failure, death or chronic kidney disease stage 4 plus within one year after transplantation, modelled using logistic regression. Discrimination and calibration were assessed in internal, temporal and external validation. Seven existing models were validated with the same cohorts. The NOTR development cohort contained 2510 patients and 823 events. The temporal validation within NOTR had 837 patients and the external validation used 31987 patients in the United States organ transplant registry. Discrimination of our full adverse outcome model was moderate in external validation (C-statistic 0.63), though somewhat better than discrimination of the seven existing prediction models (average C-statistic 0.57). The model's calibration was highly accurate. Thus, since existing adverse outcome kidney graft survival models performed poorly in a population of older deceased donors, novel models were developed and externally validated, with maximum achievable performance in a population of older deceased kidney donors. These models could assist transplant clinicians in deciding whether to accept a kidney from an older donor.Clinical epidemiolog

Renal function decline in older men and women with advanced chronic kidney disease-results from the EQUAL study

Introduction. Understanding the mechanisms underlying the differences in renal decline between men and women may improve sex-specific clinical monitoring and management. To this end, we aimed to compare the slope of renal function decline in older men and women in chronic kidney disease (CKD) Stages 4 and 5, taking into account informative censoring related to the sex-specific risks of mortality and dialysis initiation.Methods. The European QUALity Study on treatment in advanced CKD (EQUAL) study is an observational prospective cohort study in Stages 4 and 5 CKD patients >= 65years not on dialysis. Data on clinical and demographic patient characteristics were collected between April 2012 and December 2018. Estimated glomerular filtration rate (eGFR) was calculated using the CKD Epidemiology Collaboration equation. eGFR trajectory by sex was modelled using linear mixed models, and joint models were applied to deal with informative censoring.Results. We included 7801 eGFR measurements in 1682 patients over a total of 2911years of follow-up. Renal function declined by 14.0% [95% confidence interval (CI) 12.9-15.1%] on average each year. Renal function declined faster in men (16.2%/year, 95% CI 15.9-17.1%) compared with women (9.6%/year, 95% CI 6.3-12.1%), which remained largely unchanged after accounting for various mediators and for informative censoring due to mortality and dialysis initiation. Diabetes was identified as an important determinant of renal decline specifically in women.Conclusion. In conclusion, renal function declines faster in men compared with women, which remained similar after adjustment for mediators and despite a higher risk of informative censoring in men. We demonstrate a disproportional negative impact of diabetes specifically in women.Clinical epidemiolog

Prediction meets causal inference: the role of treatment in clinical prediction models

In this paper we study approaches for dealing with treatment when developing a clinical prediction model. Analogous to the estimand framework recently proposed by the European Medicines Agency for clinical trials, we propose a ‘predictimand’ framework of different questions that may be of interest when predicting risk in relation to treatment started after baseline. We provide a formal definition of the estimands matching these questions, give examples of settings in which each is useful and discuss appropriate estimators including their assumptions. We illustrate the impact of the predictimand choice in a dataset of patients with end-stage kidney disease. We argue that clearly defining the estimand is equally important in prediction research as in causal inference

Kidney failure prediction models: a comprehensive external validation study in patients with advanced CKD

Background: Various prediction models have been developed to predict the risk of kidney failure in patients with CKD. However, guideline-recommended models have yet to be compared head to head, their validation in patients with advanced CKD is lacking, and most do not account for competing risks.Methods: To externally validate 11 existing models of kidney failure, taking the competing risk of death into account, we included patients with advanced CKD from two large cohorts: the European Quality Study (EQUAL), an ongoing European prospective, multicenter cohort study of older patients with advanced CKD, and the Swedish Renal Registry (SRR), an ongoing registry of nephrology-referred patients with CKD in Sweden. The outcome of the models was kidney failure (defined as RRT-treated ESKD). We assessed model performance with discrimination and calibration.Results: The study included 1580 patients from EQUAL and 13,489 patients from SRR. The average c statistic over the 11 validated models was 0.74 in EQUAL and 0.80 in SRR, compared with 0.89 in previous validations. Most models with longer prediction horizons overestimated the risk of kidney failure considerably. The 5-year Kidney Failure Risk Equation (KFRE) overpredicted risk by 10%-18%. The four- and eight-variable 2-year KFRE and the 4-year Grams model showed excellent calibration and good discrimination in both cohorts.Conclusions: Some existing models can accurately predict kidney failure in patients with advanced CKD. KFRE performed well for a shorter time frame (2 years), despite not accounting for competing events. Models predicting over a longer time frame (5 years) overestimated risk because of the competing risk of death. The Grams model, which accounts for the latter, is suitable for longer-term predictions (4 years).Clinical epidemiolog