Background: Progression-free survival (PFS) is often used instead of the patient-relevant endpoint overall survival (OS) in cancer clinical trials. In order for PFS to be accepted as a patient-relevant outcome within the benefit assessment of pharmaceuticals in accordance with the German Social Code, Book Five (SGB V), section 35a, it has to be validated as a surrogate endpoint for OS in the relevant indication. As part of a rapid report the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – IQWiG) presented methods for surrogate endpoints validation and recommendations for correlation-based procedures. These methods include the evaluation of the certainty of conclusion of study results and the correlation between estimates of surrogate outcome and patient-relevant outcome on trial-level. The correlation is estimated by sample Pearson correlation coefficient or coefficient of determination and respective confidence interval (CI). Requirements for surrogate validation are a high correlation and a high certainty of conclusion of the study results. In case of medium correlation IQWiG methods propose applying the concept of surrogate threshold effect (STE) to determine thresholds for the estimate of the surrogate endpoint.Methods: In simulation studies we investigate the requirements for a successful surrogate validation when applying a correlation-based approach. Simulation parameters are the estimates of the surrogate and the patient-relevant outcome, the correlation between them, the number of patients and the number of studies. We analyzed different scenarios in order to figure out parameters contributing to high correlation. Furthermore, we investigate requirements of the STE method, allowing conclusions on patient-relevant endpoints by means of surrogate endpoints. Finally, in consideration of IQWiG methods we analyze the challenges of surrogate validation in practical use.Results: Both, simulations of the surrogate validation using correlation-based procedure as well as an analytical derivation show low statistical power despite a medium-sized number of studies and a high true correlation. The power for =5 studies and correlation =0.9 is below 6%. A very high true correlation of =0.95 in at least =25 studies would be required in order to preserve a power of 80%, however this scenario is considered implausible in practice. Further simulations investigating the power of the method of STE showed that only one fifth of the considered scenarios have power above 80%. However, these scenarios included parameter constellations with impractical values regarding number of studies, number of patients and effect estimate of OS. The correlation parameter as well as the parameter of the estimate of PFS barely have an impact on the power of the STE procedure.Conclusion: Our simulations show that in practical use it is quite unlikely to fulfill the condition of high correlation as defined in the rapid report of IQWiG, proposing the lower limit of confidence interval to be crucial. Despite setting the true correlation in the model to a high value, statistical power will be quite small as long as the number of studies remains low or medium which is a realistic assumption in validation of surrogate endpoints within the framework of early benefit assessment. Besides, recommendation to involve certainty of studies in the analysis remains problematic. On closer inspection of the density function of sample correlation coefficient and assuming a given true correlation we can conclude that sample correlation does not depend on the variance of the single estimates but only on sample size (representing the number of studies in the model). Therefore, patient number does not have an impact on the confidence interval of the correlation whether using weight vectors for studies or not. Application of the STE concept according to the requirements described in the rapid report appears to be rather complicated as well. We propose an alternative solution of comparing the value of STE with point estimate of the surrogate endpoint instead of its lower level of confidence interval showing low α-errors in realistic scenarios
