Pneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies

Broekhuizen, Berna D. L.

de Groot, Joris A. H.

de Wit, Niek J.

Dinant, Geert-Jan

Gonzales, Ralph

Graffelman, Aleida W.

Holm, Anette

Hopstaken, Rogier M.

Melbye, Hasse

Minnaard, Margaretha C.

Rainer, Timothy

Reitsma, Johannes B.

Schierenberg, Alwin

Steurer, Johann

van de Pol, Alma C.

van Vugt, Saskia F.

Verheij, Theo J. M.

PLoS ONE

PubMed

Background:
Pneumonia remains difficult to diagnose in primary care. Prediction models based on signs
and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of
these models is essential before implementation into routine practice. In this study all published
S&S models for prediction of pneumonia in primary care were externally validated in
the individual patient data (IPD) of previously performed diagnostic studies.
Methods and Findings:
S&S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory
tract infection and IPD for validation were identified through a systematical search.
Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia
12%) were included. Models were assessed on discrimination and calibration. Discrimination
was measured using the pooled Area Under the Curve (AUC) and delta AUC,
representing the performance of an individual model relative to the average dataset performance.
Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest
pooled AUC of 0.79 (95% CI 0.74–0.85) and 0.72 (0.68–0.76), respectively. Other models
by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs
of 0.65 (0.61–0.68), 0.64 (0.61–0.67), 0.56 (0.49–0.63) and 0.53 (0.5–0.56), respectively. A
similar ranking was present based on the delta AUCs of the models. Calibration demonstrated
close agreement of observed and predicted probabilities in the models by van Vugt
et al. and Singal et al., other models lacked such correspondence. The absence of predictors
in the IPD on dataset level hampered a systematical comparison of model performance
and could be a limitation to the study.
Conclusions:
The model by van Vugt et al. demonstrated the highest discriminative accuracy coupled
with reasonable to good calibration across the IPD of different study populations. This
model is therefore the main candidate for primary care use

Van De Pol, Alma C.

Broekhuizen, Berna D.L.

De Wit, Niek J.

Van Vugt, Saskia F.

Rainer, Timothy H.

De Groot, Joris A.H.

Verheij, Theo J.M.

NORA - Norwegian Open Research Archives

External validation of prediction models for pneumonia in primary care patients with lower respiratory tract infection: An individual patient data meta-analysis

<div>BackgroundPneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies.Methods and FindingsS&S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory tract infection and IPD for validation were identified through a systematical search. Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia 12%) were included. Models were assessed on discrimination and calibration. Discrimination was measured using the pooled Area Under the Curve (AUC) and delta AUC, representing the performance of an individual model relative to the average dataset performance. Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest pooled AUC of 0.79 (95% CI 0.74–0.85) and 0.72 (0.68–0.76), respectively. Other models by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs of 0.65 (0.61–0.68), 0.64 (0.61–0.67), 0.56 (0.49–0.63) and 0.53 (0.5–0.56), respectively. A similar ranking was present based on the delta AUCs of the models. Calibration demonstrated close agreement of observed and predicted probabilities in the models by van Vugt et al. and Singal et al., other models lacked such correspondence. The absence of predictors in the IPD on dataset level hampered a systematical comparison of model performance and could be a limitation to the study.ConclusionsThe model by van Vugt et al. demonstrated the highest discriminative accuracy coupled with reasonable to good calibration across the IPD of different study populations. This model is therefore the main candidate for primary care use.</div

Alwin Schierenberg (2503363)

Margaretha C. Minnaard (2503345)

Rogier M. Hopstaken (2503348)

Alma C. van de Pol (2503354)

Berna D. L. Broekhuizen (470781)

Niek J. de Wit (2503342)

Johannes B. Reitsma (328343)

Saskia F. van Vugt (2503366)

Aleida W. Graffelman (2503360)

Hasse Melbye (475905)

Timothy H. Rainer (621614)

Johann Steurer (5157)

Anette Holm (2503357)

Ralph Gonzales (2503351)

Geert-Jan Dinant (21827)

Joris A. H. de Groot (644365)

Theo J. M. Verheij (475904)

FigShare

External Validation of Prediction Models for Pneumonia in Primary Care Patients with Lower Respiratory Tract Infection: An Individual Patient Data Meta-Analysis

Alwin Schierenberg

Margaretha C. Minnaard

Rogier M. Hopstaken

Alma C. van de Pol

Berna D. L. Broekhuizen

Niek J. de Wit

Johannes B. Reitsma

Saskia F. van Vugt

Aleida W. Graffelman

Hasse Melbye

Timothy H. Rainer

Johann Steurer

Anette Holm

Ralph Gonzales

Geert-Jan Dinant

Joris A. H. de Groot

Theo J. M. Verheij

Crossref

Online Research @ Cardiff

External validation of prediction models for pneumonia in primary care patients with lower respiratory tract infection: an individual patient data meta-analysis

BackgroundPneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&amp;S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&amp;S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies.Methods and findingsS&amp;S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory tract infection and IPD for validation were identified through a systematical search. Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia 12%) were included. Models were assessed on discrimination and calibration. Discrimination was measured using the pooled Area Under the Curve (AUC) and delta AUC, representing the performance of an individual model relative to the average dataset performance. Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest pooled AUC of 0.79 (95% CI 0.74-0.85) and 0.72 (0.68-0.76), respectively. Other models by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs of 0.65 (0.61-0.68), 0.64 (0.61-0.67), 0.56 (0.49-0.63) and 0.53 (0.5-0.56), respectively. A similar ranking was present based on the delta AUCs of the models. Calibration demonstrated close agreement of observed and predicted probabilities in the models by van Vugt et al. and Singal et al., other models lacked such correspondence. The absence of predictors in the IPD on dataset level hampered a systematical comparison of model performance and could be a limitation to the study.ConclusionsThe model by van Vugt et al. demonstrated the highest discriminative accuracy coupled with reasonable to good calibration across the IPD of different study populations. This model is therefore the main candidate for primary care use

Minnaard, Margaretha C

Hopstaken, Rogier M

van de Pol, Alma C

Broekhuizen, Berna DL

de Wit, Niek J

Reitsma, Johannes B

van Vugt, Saskia F

Graffelman, Aleida W

Rainer, Timothy H

de Groot, Joris AH

Verheij, Theo JM

eScholarship - University of California

Background Pneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies. Methods and Findings S&S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory tract infection and IPD for validation were identified through a systematical search. Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia 12%) were included. Models were assessed on discrimination and calibration. Discrimination was measured using the pooled Area Under the Curve (AUC) and delta AUC, representing the performance of an individual model relative to the average dataset performance. Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest pooled AUC of 0.79 (95% CI 0.74-0.85) and 0.72 (0.68-0.76), respectively. Other models by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs of 0.65 (0.61-0.68), 0.64 (0.61-0.67), 0.56 (0.49-0.63) and 0.53 (0.5-0.56), respectively. A similar ranking was present based on the delta AUCs of the models. Calibration demonstrated close agreement of observed and predicted probabilities in the models by van Vugt et al. and Singal et al., other models lacked such correspondence. The absence of predictors in the IPD on dataset level hampered a systematical comparison of model performance and could be a limitation to the study. Conclusions The model by van Vugt et al. demonstrated the highest discriminative accuracy coupled with reasonable to good calibration across the IPD of different study populations. This model is therefore the main candidate for primary care use

de Vries-van Vugt, Saskia F.

Dinant, Geert Jan

de Groot, Joris A H

Verheij, Theo J M

NARCIS 

External validation of prediction models for pneumonia in primary care patients with lower respiratory tract infection : An individual patient data meta-analysis

Broekhuizen, Berna D L

University of Southern Denmark Research Output

External Validation of Prediction Models for Pneumonia in Primary Care Patients with Lower Respiratory Tract Infection:An Individual Patient Data Meta-Analysis

BACKGROUND  Pneumonia remains difficult to diagnose in primary care. Prediction models based on signs and symptoms (S&S) serve to minimize the diagnostic uncertainty. External validation of these models is essential before implementation into routine practice. In this study all published S&S models for prediction of pneumonia in primary care were externally validated in the individual patient data (IPD) of previously performed diagnostic studies.  METHODS AND FINDINGS  S&S models for diagnosing pneumonia in adults presenting to primary care with lower respiratory tract infection and IPD for validation were identified through a systematical search. Six prediction models and IPD of eight diagnostic studies (N total = 5308, prevalence pneumonia 12%) were included. Models were assessed on discrimination and calibration. Discrimination was measured using the pooled Area Under the Curve (AUC) and delta AUC, representing the performance of an individual model relative to the average dataset performance. Prediction models by van Vugt et al. and Heckerling et al. demonstrated the highest pooled AUC of 0.79 (95% CI 0.74-0.85) and 0.72 (0.68-0.76), respectively. Other models by Diehr et al., Singal et al., Melbye et al., and Hopstaken et al. demonstrated pooled AUCs of 0.65 (0.61-0.68), 0.64 (0.61-0.67), 0.56 (0.49-0.63) and 0.53 (0.5-0.56), respectively. A similar ranking was present based on the delta AUCs of the models. Calibration demonstrated close agreement of observed and predicted probabilities in the models by van Vugt et al. and Singal et al., other models lacked such correspondence. The absence of predictors in the IPD on dataset level hampered a systematical comparison of model performance and could be a limitation to the study.  CONCLUSIONS  The model by van Vugt et al. demonstrated the highest discriminative accuracy coupled with reasonable to good calibration across the IPD of different study populations. This model is therefore the main candidate for primary care use

ZORA

External validation of prediction models for pneumonia in primary care patients with lower respiratory tract infection: an individual patient data meta-analysis

Abstract

Similar works

Full text

Available Versions