Determination of the relative economic impact of different molecular-based laboratory algorithms for respiratory viral pathogen detection, including Pandemic (H1N1), using a secure web based platform

<p>Abstract</p> <p>Background</p> <p>During period of crisis, laboratory planners may be faced with a need to make operational and clinical decisions in the face of limited information. To avoid this dilemma, our laboratory utilizes a secure web based platform, Data Integration for Alberta Laboratories (DIAL) to make near real-time decisions.</p> <p>This manuscript utilizes the data collected by DIAL as well as laboratory test cost modeling to identify the relative economic impact of four proposed scenarios of testing for Pandemic H1N1 (2009) and other respiratory viral pathogens.</p> <p>Methods</p> <p>Historical data was collected from the two waves of the pandemic using DIAL. Four proposed molecular testing scenarios were generated: A) Luminex respiratory virus panel (RVP) first with/without US centers for Disease Control Influenza A Matrix gene assay (CDC-M), B) CDC-M first with/without RVP, C) RVP only, and D) CDC-M only. Relative cost estimates of different testing algorithm were generated from a review of historical costs in the lab and were based on 2009 Canadian dollars.</p> <p>Results</p> <p>Scenarios A and B had similar costs when the rate of influenza A was low (< 10%) with higher relative cost in Scenario A with increasing incidence. Scenario A provided more information about mixed respiratory virus infection as compared with Scenario B.</p> <p>Conclusions</p> <p>No one approach is applicable to all conditions. Testing costs will vary depending on the test volume, prevalence of influenza A strains, as well as other circulating viruses and a more costly algorithm involving a combination of different tests may be chosen to ensure that tests results are returned to the clinician in a quicker manner. Costing should not be the only consideration for determination of laboratory algorithms.</p

Innovative technology for web-based data management during an outbreak

Lack of automated and integrated data collection and management, and poor linkage of clinical, epidemiological and laboratory data during an outbreak can inhibit effective and timely outbreak investigation and response. This paper describes an innovative web-based technology, referred to as Web Data, developed for the rapid set-up and provision of interactive and adaptive data management during outbreak situations. We also describe the benefits and limitations of the Web Data technology identified through a questionnaire that was developed to evaluate the use of Web Data implementation and application during the 2009 H1N1 pandemic by Winnipeg Regional Health Authority and Provincial Laboratory for Public Health of Alberta. Some of the main benefits include: improved and secure data access, increased efficiency and reduced error, enhanced electronic collection and transfer of data, rapid creation and modification of the database, conversion of specimen-level to case-level data, and user-defined data extraction and query capabilities. Areas requiring improvement include: better understanding of privacy policies, increased capability for data sharing and linkages between jurisdictions to alleviate data entry duplication

Innovative technology for web-based data management during an outbreak

Lack of automated and integrated data collection and management, and poor linkage of clinical, epidemiological and laboratory data during an outbreak can inhibit effective and timely outbreak investigation and response. This paper describes an innovative web-based technology, referred to as Web Data, developed for the rapid set-up and provision of interactive and adaptive data management during outbreak situations. We also describe the benefits and limitations of the Web Data technology identified through a questionnaire that was developed to evaluate the use of Web Data implementation and application during the 2009 H1N1 pandemic by Winnipeg Regional Health Authority and Provincial Laboratory for Public Health of Alberta. Some of the main benefits include: improved and secure data access, increased efficiency and reduced error, enhanced electronic collection and transfer of data, rapid creation and modification of the database, conversion of specimen-level to case-level data, and user-defined data extraction and query capabilities. Areas requiring improvement include: better understanding of privacy policies, increased capability for data sharing and linkages between jurisdictions to alleviate data entry duplication

A Confidence-based Aberration Interpretation Framework For Outbreak Conciliation

Health surveillance can be viewed as an ongoing systematic collection, analysis, and interpretation of data for use in planning, implementation, and evaluation of a given health system, in potentially multiple spheres (ex: animal, human, environment). As we move into a sophisticated technologically advanced era, there is a need for cost-effective and efficient health surveillance methods and systems that will rapidly identify potential bioterrorism attacks and infectious disease outbreaks. The main objective of such methods and systems would be to reduce the impact of an outbreak by enabling appropriate officials to detect it quickly and implement timely and appropriate interventions. Identifying an outbreak and/or potential bioterrorism attack days to weeks earlier than traditional surveillance methods would potentially result in a reduction in morbidity, mortality, and outbreak associated economic consequences. Proposed here is a novel framework that would enable a user and/or a system to interpret the anomaly detection results generated via multiple aberration detection algorithms with some indication of confidence. A framework that takes into account the relationships between algorithms and produces an unbiased confidence measure for identification of start of an outbreak

User rating activity within KIWI: A technology for public health event monitoring and early warning signal detection

Objectives: To review user signal rating activity within the Canadian Network for Public Health Intelligence’s (CNPHI’s) Knowledge Integration using Web-based Intelligence (KIWI) technology by answering the following questions: (1) who is rating, (2) how are users rating, and (3) how well are users rating?Methods: KIWI rating data was extracted from the CNPHI platform. Zoonotic &amp; Emerging program signals with first rating occurring between January 1, 2016 and December 31, 2017 were included. Krippendorff’s alpha was used to estimate inter-rater reliability between users. A z-test was used to identify whether users tended to rate within 95% confidence interval (versus outside) the average community rating.Results: The 37 users who rated signals represented 20 organizations. 27.0% (n = 10) of users rated ≥10% of all rated signals, and their inter-rater reliability estimate was 72.4% (95% CI: 66.5-77.9%). Five users tended to rate significantly outside of the average community rating. An average user rated 58.4% of the time within the signal’s 95% CI. All users who significantly rated within the average community rating rated outside the 95% CI at least once.Discussion: A diverse community of raters participated in rating the signals. Krippendorff’s Alpha estimate revealed moderate reliability for users who rated ≥10% of signals. It was observed that inter-rater reliability increased for users with more experience rating signals.Conclusions: Diversity was observed between user ratings. It is hypothesized that rating diversity is influenced by differences in user expertise and experience, and that the number of times a user rates within and outside of a signal’s 95% CI can be used as a proxy for user expertise. The introduction of a weighted rating algorithm within KIWI that takes this into consideration could be beneficial

Development and Validation of a Standardized Tool for Prioritization of Information Sources

Purpose: To validate the utility and effectiveness of a standardized tool for prioritization of information sources for early detection of diseases.Methods: The tool was developed with input from diverse public health experts garnered through survey. Ten raters used the tool to evaluate ten information sources and reliability among raters was computed. The Proc mixed procedure with random effect statement and SAS Macros were used to compute multiple raters’ Fleiss Kappa agreement and Kendall's Coefficient of Concordance.  Results: Ten disparate information sources evaluated obtained the following composite scores: ProMed 91%; WAHID 90%; Eurosurv 87%; MediSys 85%; SciDaily 84%; EurekAl 83%; CSHB 78%; GermTrax 75%; Google 74%; and CBC 70%. A Fleiss Kappa agreement of 50.7% was obtained for ten information sources and 72.5% for a sub-set of five sources rated, which is substantial agreement validating the utility and effectiveness of the tool.  Conclusion: This study validated the utility and effectiveness of the standardized criteria tool and was used to identify five information sources suited for use by the KIWI system for a pilot project focusing on emerging and zoonotic diseases. The tool can be used in prioritizing a plethora of information sources to improve early detection of diseases.

An innovative web based system for reporting rare diseases in paediatrics

BACKGROUND: Surveillance of rare diseases in children is an important aspect of public health. Rare diseases affect thousands of children worldwide. The Canadian Paediatric Surveillance Program (CPSP) has been in existence since 1996, and provides an innovative means to undertake paediatric surveillance and increase awareness of childhood disorders that are high in disability, morbidity, mortality, and economic costs to society, despite their low frequency. Traditionally, CPSP used manual paper-based reporting on a monthly basis, which although had an impressive response rate, it had inherent longer processing times and costs associated with it. OBJECTIVES: To provide an overview and evaluate an innovative web-based system that enables seamless reporting from participants across the country providing a quick, reliable and simple mechanism for the participants to submit data while yielding better data quality, timeliness and increased efficiencies. METHODS: In 2011, a proprietary electronic CPSP (eCPSP) system was developed to provide a simple, quick and reliable reporting environment for participants. It supports both the electronic and hardcopy reporting. The analysis presented in this paper was conducted based on usage data of this system. RESULTS: The response rates of the new eCPSP were found to be very favorable with adjusted rate of 80%, which equals the baseline. Approximately 50% of online participants report the first day they receive the notification e-mail. The response time was also reduced considerably. Furthermore, there has been significant reduction in data handling related activities (by almost 70%) from estimated 690 hours per year. Finally, the number of cases reported that do not fit the study case criteria has fallen, likely because participants can now immediately access the case definition and protocol via the online system. This has reduced both staff and investigator time for case processing. CONCLUSION: The eCPSP has modernized the CPSP program from paper-based reporting to efficient online technology while maintaining the core principles of the program. This simple and intuitive approach has proven to be an efficient approach cutting response times significantly while maintaining the desired response rates

An innovative web based system for reporting rare diseases in paediatrics

Surveillance of rare diseases in children is an important aspect of public health. Rare diseases affect thousands of children worldwide. The Canadian Paediatric Surveillance Program (CPSP) has been in existence since 1996, and provides an innovative means to undertake paediatric surveillance and increase awareness of childhood disorders that are high in disability, morbidity, mortality, and economic costs to society, despite their low frequency. Traditionally, CPSP used manual paper-based reporting on a monthly basis, which although had an impressive response rate, it had inherent longer processing times and costs associated with it. The article below describes an innovative web-based system that enables seamless reporting from participants across the country providing a quick, reliable and simple mechanism for the users to submit data while yielding better data quality, timeliness and increased efficiencies. The development of such a system represents a significant advancement in the public health informatics area, building capacity for seamless and rapid data management for national surveillance

KIWI: A technology for public health event monitoring and early warning detection

Objectives: To introduce the Canadian Network for Public Health Intelligence’s new Knowledge Integration using Web-based Intelligence (KIWI) technology, and to perform preliminary evaluation of the KIWI technology using a case study. The purpose of this new technology is to support surveillance activities by monitoring unstructured data sources for the early detection and awareness of potential public health threats.Methods: A prototype of the KIWI technology, adapted for zoonotic and emerging diseases, was piloted by end-users with expertise in the field of public health and zoonotic/emerging disease surveillance. The technology was assessed using variables such as geographic coverage, user participation, and others; categorized by high-level attributes from evaluation guidelines for internet based surveillance systems. Special attention was given to the evaluation of the system’s automated sense-making algorithm, which used variables such as sensitivity, specificity, and predictive values. Event-based surveillance evaluation was not applied to its full capacity as such an evaluation is beyond the scope of this paper.Results: KIWI was piloted with user participation = 85.0% and geographic coverage within monitored sources = 83.9% of countries. The pilots, which focused on zoonotic and emerging diseases, lasted a combined total of 65 days and resulted in the collection of 3243 individual information pieces (IIP) and 2 community reported events (CRE) for processing. Ten sources were monitored during the second phase of the pilot, which resulted in 545 anticipatory intelligence signals (AIS). KIWI’s automated sense-making algorithm (SMA) had sensitivity = 63.9% (95% CI: 60.2-67.5%), specificity = 88.6% (95% CI: 87.3-89.8%), positive predictive value = 59.8% (95% CI: 56.1-63.4%), and negative predictive value = 90.3% (95% CI: 89.0-91.4%).Discussion: Literature suggests the need for internet based monitoring and surveillance systems that are customizable, integrated into collaborative networks of public health professionals, and incorporated into national surveillance activities. Results show that the KIWI technology is well posied to address some of the suggested challenges. A limitation of this study is that sample size for pilot participation was small for capturing overall readiness of integrating KIWI into regular surveillance activities.Conclusions: KIWI is a customizable technology developed within an already thriving collaborative platform used by public health professionals, and performs well as a tool for discipline-specific event monitoring and early warning signal detection