Validation of population-based disease simulation models: a review of concepts and methods

Abstract Background Computer simulation models are used increasingly to support public health research and policy, but questions about their quality persist. The purpose of this article is to review the principles and methods for validation of population-based disease simulation models. Methods We developed a comprehensive framework for validating population-based chronic disease simulation models and used this framework in a review of published model validation guidelines. Based on the review, we formulated a set of recommendations for gathering evidence of model credibility. Results Evidence of model credibility derives from examining: 1) the process of model development, 2) the performance of a model, and 3) the quality of decisions based on the model. Many important issues in model validation are insufficiently addressed by current guidelines. These issues include a detailed evaluation of different data sources, graphical representation of models, computer programming, model calibration, between-model comparisons, sensitivity analysis, and predictive validity. The role of external data in model validation depends on the purpose of the model (e.g., decision analysis versus prediction). More research is needed on the methods of comparing the quality of decisions based on different models. Conclusion As the role of simulation modeling in population health is increasing and models are becoming more complex, there is a need for further improvements in model validation methodology and common standards for evaluating model credibility

Assessing data protection and governance in health information systems: A novel methodology of Privacy and Ethics Impact and Performance Assessment (PEIPA)

Background: Data processing of health research databases often requires a Data Protection Impact Assessment to evaluate the severity of the risk and the appropriateness of measures taken to comply with the European Union (EU) General Data Protection Regulation (GDPR). We aimed to define and apply a comprehensive method for the evaluation of privacy, data governance and ethics among research networks involved in the EU Project Bridge Health. Methods: Computerised survey among associated partners of main EU Consortia, using a targeted instrument designed by the principal investigator and progressively refined in collaboration with an international advisory panel. Descriptive measures using the percentage of adoption of privacy, data governance and ethical principles as main endpoints were used for the analysis and interpretation of the results. Results: A total of 15 centres provided relevant information on the processing of sensitive data from 10 European countries. Major areas of concern were noted for: data linkage (median, range of adoption: 45%, 30%-80%), access and accuracy of personal data (50%, 0%-100%) and anonymisation procedures (56%, 11%-100%). A high variability was noted in the application of privacy principles. Conclusions: A comprehensive methodology of Privacy and Ethics Impact and Performance Assessment was successfully applied at international level. The method can help implementing the GDPR and expanding the scope of Data Protection Impact Assessment, so that the public benefit of the secondary use of health data could be well balanced with the respect of personal privacy