Use of outcomes to evaluate surveillance systems for bioterrorist attacks

<p>Abstract</p> <p>Background</p> <p>Syndromic surveillance systems can potentially be used to detect a bioterrorist attack earlier than traditional surveillance, by virtue of their near real-time analysis of relevant data. Receiver operator characteristic (ROC) curve analysis using the area under the curve (AUC) as a comparison metric has been recommended as a practical evaluation tool for syndromic surveillance systems, yet traditional ROC curves do not account for timeliness of detection or subsequent time-dependent health outcomes.</p> <p>Methods</p> <p>Using a decision-analytic approach, we predicted outcomes, measured in lives, quality adjusted life years (QALYs), and costs, for a series of simulated bioterrorist attacks. We then evaluated seven detection algorithms applied to syndromic surveillance data using outcomes-weighted ROC curves compared to simple ROC curves and timeliness-weighted ROC curves. We performed sensitivity analyses by varying the model inputs between best and worst case scenarios and by applying different methods of AUC calculation.</p> <p>Results</p> <p>The decision analytic model results indicate that if a surveillance system was successful in detecting an attack, and measures were immediately taken to deliver treatment to the population, the lives, QALYs and dollars lost could be reduced considerably. The ROC curve analysis shows that the incorporation of outcomes into the evaluation metric has an important effect on the apparent performance of the surveillance systems. The relative order of performance is also heavily dependent on the choice of AUC calculation method.</p> <p>Conclusions</p> <p>This study demonstrates the importance of accounting for mortality, morbidity and costs in the evaluation of syndromic surveillance systems. Incorporating these outcomes into the ROC curve analysis allows for more accurate identification of the optimal method for signaling a possible bioterrorist attack. In addition, the parameters used to construct an ROC curve should be given careful consideration.</p

Enhancing community health through patient navigation, advocacy, and social support: A community health navigator pilot study

Background: The healthcare system is complex and difficult to navigate, particularly for patients with multiple chronic conditions and complex care plans. Patient adherence to care plans and patient health outcomes can be negatively impacted by language, financial, and other social barriers. Community Health Navigators (CHNs) are community members that are hired and trained to navigate the healthcare system, who work with patients to overcome barriers to care and support patient self-management by providing services tailored to needs. While these types of interventions can improve access to care in other settings, they are not well studied in Canada nor in Canadian primary care settings. Objective: For this pilot study, we aimed to determine the feasibility of a CHN intervention for patients with multiple chronic conditions. Our secondary objective was to assess the potential impact of a CHN intervention on patient-reported outcome measures. Methods: We used an observational single arm pre-post study design. Using interviewer-administered patient surveys, we assessed patient-reported outcomes at baseline (pre-enrolment), and 6-months and 12-months post-enrolment. The survey included instruments to assess quality of life (EQ-5D-5L), patient chronic disease care experience (PACIC), social support (mMOS-SS), and cost-related adherence to care (i.e. financial security to pay for care-related costs). Descriptive analysis was performed on survey data, and the sample was restricted to participants who completed both follow-up surveys (6- and 12-month). &nbsp; Results: Of the 21 participants enrolled in our pilot study, the mean age was 61.3 years, 56% had an annual household income below $30,000, and 68% were born outside of Canada. The three most common conditions reported were hypertension (77%), diabetes (59%), and back problems (55%). The mean number of conditions a patient reported was 5.4 (SD 2.3, range 3-11). Of the sample enrolled, 14 (67%) patients completed both follow-up surveys. Mean social support (scale: 0-100), was 56, 68, and 75 at baseline, 6, and 12 months, respectively—indicating a potential increase in social support after the intervention. Mean self-ranked health (scale: 0-100) did not change over time. Mean patient experience with chronic disease care (scale: 1-3) was 2.01 at baseline; 2.24 at 6 months, and 1.89 at 12 months. &nbsp;The proportion of patients who reported no difficulty paying for medical expenses increased from 36% at baseline to 79% at 6 months and 86% at 12 months. In other words, fewer patients reported difficulty paying for medical expenses at 6 months and at 12 months. Results presented here are preliminary; further analysis is underway which will include analysis of health outcomes using administrative data, statistical tests of survey data (where appropriate), and qualitative analysis of interview data. Conclusions: CHNs may improve patients’ social and financial support and satisfaction with care. Our pilot study demonstrates that a CHN intervention is feasible to implement in primary care for patients with multiple chronic conditions. These findings informed a large ongoing cluster-randomized pragmatic trial

Assessing the impact of Connect 2 Care on the residential stability of homeless and vulnerably housed clients

Background Certain kinds of housing instability, such as foreclosure and homelessness, have been associated with poorer physical and mental health. The Connect 2 Care (C2C) program targets medically complex individuals who are unstably housed, primarily aimed at reducing acute care utilization and connecting clients to appropriate community-based care. However, because housing status is a fundamental determinant of health, the team also assists clients in finding permanent housing. As the C2C program aims to improve the health of its clients, we hope that this intervention positively impacts the housing stability of clients. Objective To determine whether the C2C program is effective in reducing factors of housing instability, such as the frequency of housing moves made, and time spent in unstable housing (such as shelters or sleeping outside). Methods C2C clients were asked to participate in 6- and 12- month follow-up surveys with a member of the research team. During both surveys, participants were prompted to describe their housing history using the Residential Time-Line Follow-Back (rTLFB) inventory. Starting at six months prior to their intake into C2C, participants created a twelve- to eighteen-month timeline that detailed their residential locations and number of housing transitions. Location descriptions provided by clients were categorized as stable, temporary, institutional, or literal homelessness. The number of housing transitions and the proportion of time spent in each housing category were then calculated for each individual. Changes in proportion of time spent over three unique time periods were evaluated using Wilcoxon’s paired rank test with Holm’s multiplicity correction. Results Since September 2018, housing data was collected from 100 unique clients. In comparing the six months preceding C2C intake with the six-to-twelve months after C2C intake, significant reductions in the amount of time spent in literal homelessness (p &lt; 0.001) and reductions in the number of housing changes (p = 0.014) were observed. Discussion Housing stability for C2C clients improved after enrolment in the program. This study was potentially limited by incomplete sampling of the C2C population. Based on our findings, further research should be conducted in evaluating the relationship between increases in housing stability and increases of health status. Acknowledgements The C2C research team thanks Alberta Innovates and the Canadian Institute of Health Research for their financial support. The authors have no conflict of interests to state

Association rule mining to identify potential under-coding of conditions in the problem list in primary care electronic medical records

Introduction The problem list of a patient’s primary care electronic medical record (EMR) generally reflects their important medical conditions. We will use association rule mining to assess between-provider and between-clinic variation in the coding of select conditions in the EMR problem list, in order to identify possible under-coding outliers. Objectives and Approach EMR data from participating clinics in the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) will be used, with a focus on three commonly-occurring conditions (hypertension, diabetes, and depression). Association rule mining will be used to develop association rules between these conditions and other clinical information available in the EMR, such as other diagnoses in the problem list, billing codes, medications, and laboratory results (e.g., a rule of “diabetic medication→diabetes” indicates that patients prescribed a diabetic medication are likely to have diabetes in the problem list). Under-coding outliers at the provider and clinic levels will be identified by comparing rule enforcement. Results Results from this work in progress will be presented at the conference. An estimated 270 clinics, 1340 providers, and 1.8 million patients will be included from the CPCSSN database. Rule ‘confidence’ will be used to identify outliers; the confidence of a rule X→Y is the proportion of individuals with X who also have Y (Pr(Y|X)). For example, we may find that on average 80% of patients prescribed a diabetic medication will also have a diagnosis of diabetes in the problem list (average confidence of 80%), but an outlier clinic may have a confidence of 40%; this low rule confidence may indicate under-coding of diabetes in the problem list. Confounding by patient demographics (e.g., age, sex, urban/rural) will be assessed and adjusted for, if necessary. Conclusion/Implications This work examines a novel method to identify potential under-coding in the EMR problem list. Providers/clinics could use this information to update patients’ problem list or inform quality improvement interventions. Researchers using primary care EMR data need to be aware of potential under-coding and take steps to mitigate the effects

Examining the Usefulness of Patient Documentation Forms as a Tool for Community Health Navigators: Findings from the ENCOMPASS Pilot Study

Introduction | Effective documentation of patient encounters may influence Community Health Navigators’ (CHNs) success in providing support to patients as well as provide a data source to examine CHN practices. The ENhancing COMmunity health through Patient navigation, Advocacy, and Social Support (ENCOMPASS) study, based in partnership between the University of Calgary and the Mosaic Primary Care Network (MPCN) is evaluating a CHN program to determine whether CHNs improve outcomes for patients with multiple chronic conditions. CHNs support their patients by helping them navigate the health system, connect to community resources, and access culturally appropriate support. The purpose of this study was to examine the quality and usefulness of CHN-patient documentation forms used in the ENCOMPASS pilot study (i.e., Initial Action Planning Form, Follow-up Action Planning Form, Patient Encounter Form, all implemented on the REDCap platform) and revise the documentation process using co-design with the end user. Methods | An iterative co-design quality improvement process was employed across three phases. First, content analyses were conducted on the Patient Encounter Form notes to examine how CHNs were using the forms and how they were documenting their activities. Second, a survey was distributed to CHNs to gather their perspectives about their experiences with the REDCap platform and the three forms. Third, a working group, consisting of four CHNs, met twice with research team members to discuss barriers to use and opportunities for improvement. Results | The REDCap platform and the three CHN-patient encounter forms did not adequately meet the needs of the CHNs. Content analysis revealed significant variation in how the Patient Encounter Form was utilized and various form sections were not completed as intended. In the survey, CHNs reported that the documentation experience was not satisfactory and the training that they had received to date was insufficient. The CHN working group suggested changes to the interface with the REDCap platform and form structure. Revisions were made based on these suggestions, and approved by the working group. Conclusions | The approved changes to REDCap and the three forms will be implemented and introduced to the CHN team. The research team will develop a patient encounter documentation guidelines document and will provide all members of the CHN team with the opportunity to receive re-training. These changes will be reviewed with the CHNs to continue the iterative quality improvement process. Prior to final implementation, consultation with the Clinical Research Unit administrators on the feasibility of the revisions made to the forms and interface with the REDCap platform will be held. The results of this study have the potential to provide a better overall experience for CHNs in the ENCOMPASS program and enhance their work with patients

Impact of the Choosing Wisely Canada recommendations on potentially inappropriate antibiotic prescribing in emergency medicine across Alberta, Canada: An interrupted time-series analysis.

Objectives In Alberta, Canada, we quantified the rate of potentially inappropriate oral antibiotic prescribing in emergency departments for viral infections or conditions not likely requiring antibiotics from 2010-2020 and assessed the impact of two Choosing Wisely Canada (CWC) campaigns (2015/2016 and 2018) discouraging inappropriate antibiotic prescribing in emergency medicine. Approach We linked emergency department adult and pediatric records from the National Ambulatory Care Reporting System and medication dispensations from community-based pharmacies in the Pharmaceutical Information Network. From January 2010 to February 2020, we identified emergency department visits for 5 conditions that were potentially inappropriately treated using antibiotics per CWC recommendations (bronchitis, asthma, bronchiolitis, pharyngitis, and acute otitis media). We used an interrupted time series design to detect changes in antibiotic prescribing by fitting Autoregressive Integrated Moving Average (ARIMA) models to account for secular trends and seasonality, allowing for change in slopes to measure the effect of each CWC intervention. Results Antibiotics were commonly prescribed in emergency departments for bronchitis (proportion of visits with antibiotics: 57%) and asthma (22%) in adults; bronchiolitis in children (43%); pharyngitis (39%) and acute otitis media (54%) in adults and children. Based on visual inspection, the proportion of emergency department visits for each condition where antibiotics were dispensed remained relatively consistent. The ARIMA models demonstrated mixed impacts on potentially inappropriate antibiotic prescribing associated with two interruptions: the 2015/2016 CWC recommendations and subsequent 2018 CWC Using Antibiotics Wisely campaign. Following each interruption, antibiotic prescribing was slightly reduced for bronchitis (-1.0%/year,p=0.03; -4.4%/year,p=0.004, respectively) and bronchiolitis (not significant) (-0.7%/year,p=0.57; -2.5%/year,p=0.34), but unchanged for asthma (-0.6%/year,p=0.30; 0.7%/year,p=0.74) and pharyngitis (0.0%/year,p=0.95; -0.2%/year,p=0.93), and slightly increased for acute otitis media (not significant) (1.4%/year,p=0.07; 5.9%/year,p=0.052). Conclusion Rates of potentially inappropriate antibiotic prescribing remained constant over the past 10 years in Alberta. Campaigns to rethink antibiotic use in emergency medicine may have resulted in small decreases in antibiotic use; however, further initiatives building upon existing campaigns are required to substantially reduce rates of inappropriate antibiotic prescribing

Linking primary care EMR data and administrative data in Alberta, Canada: experiences, challenges, and potential solutions

Introduction Administrative data are commonly used for a variety of secondary purposes. Although they lack clinical detail and risk factor information, linkage to primary care electronic medical records (EMR) could fill this gap. Primary care EMRs are a relatively new data source available in Alberta and thus, EMR-administrative linkages are novel. Objectives and Approach To describe the process undertaken for linking de-identified primary care EMR data from two regional Alberta networks of the Canadian Primary Care Sentinel Surveillance Network (CPCSSN) with administrative data (hospital admissions, emergency department visits, pharmacy information) from Alberta Health Services Analytics, specifically as it relates to a study on patients with complex, chronic diseases. As this linkage process is new in Alberta, we will describe the challenges encountered and possible solutions to inform future data linkage for research studies. Results Linkage steps: 1) approval from research ethics board and individual CPCSSN providers as data custodians; 2) notify Privacy Commissioner on behalf of custodian; 3) send linking key (CPCSSN patient ID, EMR ID) from regional database to Analytics; 4) send linking files (patient personal health number [PHN], EMR ID) from custodian’s EMR system to Analytics; 5) match unique EMR ID from linking key and clinic linking files; 6) PHN from clinic linking file mapped to administrative data; 7) data de-identified before transferring to secure repository; administrative data matched to EMR data using CPCSSN ID. Challenges: obtaining individual provider consent for each study; sampling bias; delays/issues generating clinic linkage file; mismatch between patients in clinic \& regional linking files. Current and potential solutions will be discussed during the presentation. Conclusion/Implications As primary care EMR and administrative data become more routinely linked and accepted, the process will become more efficient and streamlined. These data will contribute to a better understanding of patients and their care in Alberta

Reporting of Model Performance and Statistical Methods in Studies That Use Machine Learning to Develop Clinical Prediction Models: Protocol for a Systematic Review

BACKGROUND: With the growing excitement of the potential benefits of using machine learning and artificial intelligence in medicine, the number of published clinical prediction models that use these approaches has increased. However, there is evidence (albeit limited) that suggests that the reporting of machine learning-specific aspects in these studies is poor. Further, there are no reviews assessing the reporting quality or broadly accepted reporting guidelines for these aspects. OBJECTIVE: This paper presents the protocol for a systematic review that will assess the reporting quality of machine learning-specific aspects in studies that use machine learning to develop clinical prediction models. METHODS: We will include studies that use a supervised machine learning algorithm to develop a prediction model for use in clinical practice (ie, for diagnosis or prognosis of a condition or identification of candidates for health care interventions). We will search MEDLINE for studies published in 2019, pseudorandomly sort the records, and screen until we obtain 100 studies that meet our inclusion criteria. We will assess reporting quality with a novel checklist developed in parallel with this review, which includes content derived from existing reporting guidelines, textbooks, and consultations with experts. The checklist will cover 4 key areas where the reporting of machine learning studies is unique: modelling steps (order and data used for each step), model performance (eg, reporting the performance of each model compared), statistical methods (eg, describing the tuning approach), and presentation of models (eg, specifying the predictors that contributed to the final model). RESULTS: We completed data analysis in August 2021 and are writing the manuscript. We expect to submit the results to a peer-reviewed journal in early 2022. CONCLUSIONS: This review will contribute to more standardized and complete reporting in the field by identifying areas where reporting is poor and can be improved. TRIAL REGISTRATION: PROSPERO International Prospective Register of Systematic Reviews CRD42020206167; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=206167. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR1-10.2196/30956

Consensus Recommendations for Sick Day Medication Guidance for People With Diabetes, Kidney, or Cardiovascular Disease:A Modified Delphi Process

Rationale &amp; Objective: Sick day medication guidance (SDMG) involves withholding or adjusting specific medications in the setting of acute illnesses that could contribute to complications such as hypotension, acute kidney injury (AKI), or hypoglycemia. We sought to achieve consensus among clinical experts on recommendations for SDMG that could be studied in future intervention studies. Study Design: A modified Delphi process following guidelines for conducting and reporting Delphi studies. Setting &amp; Participants: An international group of clinicians with expertise relevant to SDMG was recruited through purposive and snowball sampling. A scoping review of the literature was presented, followed by 3 sequential rounds of development, refinement, and voting on recommendations. Meetings were held virtually and structured to allow the participants to provide their input and rapidly prioritize and refine ideas.Outcome: Opinions of participants were measured as the percentage who agreed with each recommendation, whereas consensus was defined as &gt;75% agreement. Analytical Approach: Quantitative data were summarized using counts and percentages. A qualitative content analysis was performed to capture the context of the discussion around recommendations and any additional considerations brought forward by participants. Results: The final panel included 26 clinician participants from 4 countries and 10 clinical disciplines. Participants reached a consensus on 42 specific recommendations: 5 regarding the signs and symptoms accompanying volume depletion that should trigger SDMG; 6 regarding signs that should prompt urgent contact with a health care provider (including a reduced level of consciousness, severe vomiting, low blood pressure, presence of ketones, tachycardia, and fever); and 14 related to scenarios and strategies for patient self-management (including frequent glucose monitoring, checking ketones, fluid intake, and consumption of food to prevent hypoglycemia). There was consensus that renin-angiotensin system inhibitors, diuretics, nonsteroidal anti-inflammatory drugs, sodium/glucose cotransporter 2 inhibitors, and metformin should be temporarily stopped. Participants recommended that insulin, sulfonylureas, and meglitinides be held only if blood glucose was low and that basal and bolus insulin be increased by 10%-20% if blood glucose was elevated. There was consensus on 6 recommendations related to the resumption of medications within 24-48 hours of the resolution of symptoms and the presence of normal patterns of eating and drinking. Limitations: Participants were from high-income countries, predominantly Canada. Findings may not be generalizable to implementation in other settings. Conclusions: A multidisciplinary panel of clinicians reached a consensus on recommendations for SDMG in the presence of signs and symptoms of volume depletion, as well as self-management strategies and medication instructions in this setting. These recommendations may inform the design of future trials of SDMG strategies.</p

Consensus Recommendations for Sick Day Medication Guidance for People With Diabetes, Kidney, or Cardiovascular Disease : A Modified Delphi Process

Acknowledgements We would like to thank the clinical experts who participated in the modified Delphi panel and provided their knowledge and expertise. We also thank Sarah Gil and Terry Smith for technical assistance.Peer reviewedproo