Approaches to the evaluation of outbreak detection methods

BACKGROUND: An increasing number of methods are being developed for the early detection of infectious disease outbreaks which could be naturally occurring or as a result of bioterrorism; however, no standardised framework for examining the usefulness of various outbreak detection methods exists. To promote comparability between studies, it is essential that standardised methods are developed for the evaluation of outbreak detection methods. METHODS: This analysis aims to review approaches used to evaluate outbreak detection methods and provide a conceptual framework upon which recommendations for standardised evaluation methods can be based. We reviewed the recently published literature for reports which evaluated methods for the detection of infectious disease outbreaks in public health surveillance data. Evaluation methods identified in the recent literature were categorised according to the presence of common features to provide a conceptual basis within which to understand current approaches to evaluation. RESULTS: There was considerable variation in the approaches used for the evaluation of methods for the detection of outbreaks in public health surveillance data, and appeared to be no single approach of choice. Four main approaches were used to evaluate performance, and these were labelled the Descriptive, Derived, Epidemiological and Simulation approaches. Based on the approaches identified, we propose a basic framework for evaluation and recommend the use of multiple approaches to evaluation to enable a comprehensive and contextualised description of outbreak detection performance. CONCLUSION: The varied nature of performance evaluation demonstrated in this review supports the need for further development of evaluation methods to improve comparability between studies. Our findings indicate that no single approach can fulfil all evaluation requirements. We propose that the cornerstone approaches to evaluation identified provide key contributions to support internal and external validity and comparability of study findings, and suggest these be incorporated into future recommendations for performance assessment

Neonatal-onset multisystem inflammatory disease responsive to interleukin-1 beta inhibition

BACKGROUND:Neonatal-onset multisystem inflammatory disease is characterized by fever, urticarial rash, aseptic meningitis, deforming arthropathy, hearing loss, and mental retardation. Many patients have mutations in the cold-induced autoinflammatory syndrome 1 (CIAS1) gene, encoding cryopyrin, a protein that regulates inflammation.METHODS:We selected 18 patients with neonatal-onset multisystem inflammatory disease (12 with identifiable CIAS1 mutations) to receive anakinra, an interleukin-1-receptor antagonist (1 to 2 mg per kilogram of body weight per day subcutaneously). In 11 patients, anakinra was withdrawn at three months until a flare occurred. The primary end points included changes in scores in a daily diary of symptoms, serum levels of amyloid A and C-reactive protein, and the erythrocyte sedimentation rate from baseline to month 3 and from month 3 until a disease flare.RESULTS:All 18 patients had a rapid response to anakinra, with disappearance of rash. Diary scores improved (P<0.001) and serum amyloid A (from a median of 174 mg to 8 mg per liter), C-reactive protein (from a median of 5.29 mg to 0.34 mg per deciliter), and the erythrocyte sedimentation rate decreased at month 3 (all P<0.001), and remained low at month 6. Magnetic resonance imaging showed improvement in cochlear and leptomeningeal lesions as compared with baseline. Withdrawal of anakinra uniformly resulted in relapse within days; retreatment led to rapid improvement. There were no drug-related serious adverse events.CONCLUSIONS:Daily injections of anakinra markedly improved clinical and laboratory manifestations in patients with neonatal-onset multisystem inflammatory disease, with or without CIAS1 mutations

Success Factors of European Syndromic Surveillance Systems: A Worked Example of Applying Qualitative Comparative Analysis

Introduction: Syndromic surveillance aims at augmenting traditional public health surveillance with timely information. To gain a head start, it mainly analyses existing data such as from web searches or patient records. Despite the setup of many syndromic surveillance systems, there is still much doubt about the benefit of the approach. There are diverse interactions between performance indicators such as timeliness and various system characteristics. This makes the performance assessment of syndromic surveillance systems a complex endeavour. We assessed if the comparison of several syndromic surveillance systems through Qualitative Comparative Analysis helps to evaluate performance and identify key success factors. Materials and Methods: We compiled case-based, mixed data on performance and characteristics of 19 syndromic surveillance systems in Europe from scientific and grey literature and from site visits. We identified success factors by applying crisp-set Qualitative Comparative Analysis. We focused on two main areas of syndromic surveillance application: seasonal influenza surveillance and situational awareness during different types of potentially health threatening events. Results: We found that syndromic surveillance systems might detect the onset or peak of seasonal influenza earlier if they analyse non-clinical data sources. Timely situational awareness during different types of events is supported by an automated syndromic surveillance system capable of analysing multiple syndromes. To our surprise, the analysis of multiple data sources was no key success factor for situational awareness. Conclusions: We suggest to consider these key success factors when designing or further developing syndromic surveillance systems. Qualitative Comparative Analysis helped interpreting complex, mixed data on small-N cases and resulted in concrete and practically relevant findings

Interviewer BMI effects on under- and over-reporting of restrained eating: evidence from a national Dutch face-to-face survey and a postal follow-up

Contains fulltext : 102650pub.pdf (publisher's version ) (Open Access)Objectives To determine the effect of interviewer BMI on self-reported restrained eating in a face-to-face survey and to examine under- and over-reporting using the face-to face study and a postal follow-up. Methods A sample of 1,212 Dutch adults was assigned to 98 interviewers with different BMI who administered an eating questionnaire. To further evaluate misreporting a mail follow-up was conducted among 504 participants. Data were analyzed using two-level hierarchical models. Results Interviewer BMI had a positive effect on restrained eating. Normal weight and pre-obese interviewers obtained valid responses, underweight interviewers stimulated underreporting whereas obese interviewers triggered overreporting. Conclusion In face-to-face interviews self-reported dietary restraint is distorted by interviewer BMI. This result has implications for public health surveys, the more so given the expanding obesity epidemic.5 p

Telephone Triage Service Data for Detection of Influenza-Like Illness

Background: Surveillance for influenza and influenza-like illness (ILI) is important for guiding public health prevention programs to mitigate the morbidity and mortality caused by influenza, including pandemic influenza. Nontraditional sources of data for influenza and ILI surveillance are of interest to public health authorities if their validity can be established. Methods/Principal Findings: National telephone triage call data were collected through automated means for purposes of syndromic surveillance. For the 17 states with at least 500,000 inhabitants eligible to use the telephone triage services, call volume for respiratory syndrome was compared to CDC weekly number of influenza isolates and percentage of visits to sentinel providers for ILI. The degree to which the call data were correlated with either CDC viral isolates or sentinel provider percentage ILI data was highly variable among states. Conclusions: Telephone triage data in the U.S. are patchy in coverage and therefore not a reliable source of ILI surveillance data on a national scale. However, in states displaying a higher correlation between the call data and the CDC data, call data may be useful as an adjunct to state-level surveillance data, for example at times when sentinel surveillance is not in operation or in areas where sentinel provider coverage is considered insufficient. Sufficient population coverage, a specific ILI syndrome definition, and the use of a threshold of percentage of calls that are for ILI would likely improve the utility of such data for ILI surveillance purposes

Value of syndromic surveillance within the Armed Forces for early warning during a dengue fever outbreak in French Guiana in 2006

<p>Abstract</p> <p>Background</p> <p>A dengue fever outbreak occured in French Guiana in 2006. The objectives were to study the value of a syndromic surveillance system set up within the armed forces, compared to the traditional clinical surveillance system during this outbreak, to highlight issues involved in comparing military and civilian surveillance systems and to discuss the interest of syndromic surveillance for public health response.</p> <p>Methods</p> <p>Military syndromic surveillance allows the surveillance of suspected dengue fever cases among the 3,000 armed forces personnel. Within the same population, clinical surveillance uses several definition criteria for dengue fever cases, depending on the epidemiological situation. Civilian laboratory surveillance allows the surveillance of biologically confirmed cases, within the 200,000 inhabitants.</p> <p>Results</p> <p>It was shown that syndromic surveillance detected the dengue fever outbreak several weeks before clinical surveillance, allowing quick and effective enhancement of vector control within the armed forces. Syndromic surveillance was also found to have detected the outbreak before civilian laboratory surveillance.</p> <p>Conclusion</p> <p>Military syndromic surveillance allowed an early warning for this outbreak to be issued, enabling a quicker public health response by the armed forces. Civilian surveillance system has since introduced syndromic surveillance as part of its surveillance strategy. This should enable quicker public health responses in the future.</p

British Valued Life Activities Scale [British VLAs]

Background The Valued Life Activities Scale (VLAs) measures difficulty in daily activities and social participation. With various versions involving a different number of items, we have linguistically and culturally adopted the full VLAs (33-items) and psychometrically tested it in adults with rheumatic and musculoskeletal diseases in the United Kingdom. Methods Participants with Rheumatoid Arthritis, Ankylosing Spondylitis, Chronic Pain/ Fibromyalgia, Chronic Hand/ Upper Limb Conditions, Osteoarthritis, Systemic Lupus, Systemic Sclerosis and Primary Sjogren’s Syndrome were recruited from out-patient clinics in National Health Service Hospitals, General Practice and patient organisations in the UK. Phase1 involved linguistic and cultural adaptation: forward translation to British English; synthesis; expert panel review and cognitive debriefing interviews. In Phase2 participants completed postal questionnaires to assess internal construct validity using (i) Confirmatory Factor Analysis (CFA) (ii) Mokken scaling and (iii) Rasch model. Results Responders (n = 1544) had mean age of 59 years (SD13.3) and 77.2% women. A CFA failed to support a total score from the 33-items (Chi Square 3552:df 464: p < 0.0001). Mokken scaling indicated a strong non-parametric association between items. Fit to the Rasch model indicated that the VLAs was characterised by multidimensionality and item misfit, which may have been influenced by clusters of residual item correlations. An item banking approach resolved a 25-item calibrated set whose application could accommodate the ‘does not apply to me’ response option. Conclusions The UK version of the VLAs failed to satisfy classical and modern psychometric standards for complete item sets. However, as the scale is not usually applied in complete format, an item bank approach calibrated 25 items with fit to the Rasch model. Suitable Computer Adaptive Testing (CAT) software could implement the item set, giving patients the choice of whether an item applies to them, or not

Meeting the International Health Regulations (2005) surveillance core capacity requirements at the subnational level in Europe: the added value of syndromic surveillance

BACKGROUND: The revised World Health Organization's International Health Regulations (2005) request a timely and all-hazard approach towards surveillance, especially at the subnational level. We discuss three questions of syndromic surveillance application in the European context for assessing public health emergencies of international concern: (i) can syndromic surveillance support countries, especially the subnational level, to meet the International Health Regulations (2005) core surveillance capacity requirements, (ii) are European syndromic surveillance systems comparable to enable cross-border surveillance, and (iii) at which administrative level should syndromic surveillance best be applied? DISCUSSION: Despite the ongoing criticism on the usefulness of syndromic surveillance which is related to its clinically nonspecific output, we demonstrate that it was a suitable supplement for timely assessment of the impact of three different public health emergencies affecting Europe. Subnational syndromic surveillance analysis in some cases proved to be of advantage for detecting an event earlier compared to national level analysis. However, in many cases, syndromic surveillance did not detect local events with only a small number of cases. The European Commission envisions comparability of surveillance output to enable cross-border surveillance. Evaluated against European infectious disease case definitions, syndromic surveillance can contribute to identify cases that might fulfil the clinical case definition but the approach is too unspecific to comply to complete clinical definitions. Syndromic surveillance results still seem feasible for comparable cross-border surveillance as similarly defined syndromes are analysed. We suggest a new model of implementing syndromic surveillance at the subnational level. In this model, syndromic surveillance systems are fine-tuned to their local context and integrated into the existing subnational surveillance and reporting structure. By enhancing population coverage, events covering several jurisdictions can be identified at higher levels. However, the setup of decentralised and locally adjusted syndromic surveillance systems is more complex compared to the setup of one national or local system. SUMMARY: We conclude that syndromic surveillance if implemented with large population coverage at the subnational level can help detect and assess the local and regional effect of different types of public health emergencies in a timely manner as required by the International Health Regulations (2005)