A simulation study to evaluate the performance of five statistical monitoring methods when applied to different time-series components in the context of control programs for endemic diseases

Abstract

Disease monitoring and surveillance play a crucial role in control and eradication programs, as it is important to track implemented strategies in order to reduce and/or eliminate a specific disease. The objectives of this study were to assess the performance of different statistical monitoring methods for endemic disease control program scenarios, and to explore what impact of variation (noise) in the data had on the performance of these monitoring methods. We simulated 16 different scenarios of changes in weekly sero-prevalence. The changes included different combinations of increases, decreases and constant sero-prevalence levels (referred as events). Two space-state models were used to model the time series, and different statistical monitoring methods (such as univariate process control algorithms-Shewart Control Chart, Tabular Cumulative Sums, and the V-mask- and monitoring of the trend component-based on 99% confidence intervals and the trend sign) were tested. Performance was evaluated based on the number of iterations in which an alarm was raised for a given week after the changes were introduced. Results revealed that the Shewhart Control Chart was better at detecting increases over decreases in sero-prevalence, whereas the opposite was observed for the Tabular Cumulative Sums. The trend-based methods detected the first event well, but performance was poorer when adapting to several consecutive events. The V-Mask method seemed to perform most consistently, and the impact of noise in the baseline was greater for the Shewhart Control Chart and Tabular Cumulative Sums than for the V-Mask and trend-based methods. The performance of the different statistical monitoring methods varied when monitoring increases and decreases in disease sero-prevalence. Combining two of more methods might improve the potential scope of surveillance systems, allowing them to fulfill different objectives due to their complementary advantages