Inter-Rater Reliability of Statistics Based on Reconstructed Individual Patient Data from Published Kaplan-Meier Curves

Introduction: Time-to-event outcomes include two elements: an indicator variable for whether the event has taken place, and the length of time from some origin point to the occurrence of the event of interest. Due to the complexity of these data, secondary analysis methods, such as indirect comparisons and meta-analysis, are easier to perform when individual-level patient data (IPD) is available. Objectives: In 2021, an R package IPDfromKM was published, which contains an algorithm for reconstructing IPD from a Kaplan-Meier graph. The current research aimed to investigate the reproducibility of the IPDfromKM algorithm. Methods: Three statisticians (MS, LS, CW) from the University of Nebraska Medical Center Department of Biostatistics independently generated reconstructed IPD for a sample of published Kaplan-Meier curves from peer-reviewed research journals. A sample of survival metrics were collected from the reconstructed IPD datasets using the IPDfromKM package, and then compared for inter-rater reliability with the intraclass correlation coefficient (ICC). Results: Eleven Kaplan-Meier curves from five recently published journal articles were selected. The absolute agreement for survival time estimates was calculated to have an ICC of 0.967 (95% CI, 0.946, 0.981), demonstrating an excellent level of agreement. Agreement for survival probability estimates was also excellent, with an ICC of 0.983 (95% CI, 0.973, 0.99). Conclusions: The high level of inter-rater reliability of the reconstructed IPD datasets showed that the IPDfromKM algorithm provides a reproducible reconstruction of the actual survival dat