Automatic Detection of Hypoglycemic Events From the Electronic Health Record Notes of Diabetes Patients: Empirical Study

BACKGROUND: Hypoglycemic events are common and potentially dangerous conditions among patients being treated for diabetes. Automatic detection of such events could improve patient care and is valuable in population studies. Electronic health records (EHRs) are valuable resources for the detection of such events. OBJECTIVE: In this study, we aim to develop a deep-learning-based natural language processing (NLP) system to automatically detect hypoglycemic events from EHR notes. Our model is called the High-Performing System for Automatically Detecting Hypoglycemic Events (HYPE). METHODS: Domain experts reviewed 500 EHR notes of diabetes patients to determine whether each sentence contained a hypoglycemic event or not. We used this annotated corpus to train and evaluate HYPE, the high-performance NLP system for hypoglycemia detection. We built and evaluated both a classical machine learning model (ie, support vector machines [SVMs]) and state-of-the-art neural network models. RESULTS: We found that neural network models outperformed the SVM model. The convolutional neural network (CNN) model yielded the highest performance in a 10-fold cross-validation setting: mean precision=0.96 (SD 0.03), mean recall=0.86 (SD 0.03), and mean F1=0.91 (SD 0.03). CONCLUSIONS: Despite the challenges posed by small and highly imbalanced data, our CNN-based HYPE system still achieved a high performance for hypoglycemia detection. HYPE can be used for EHR-based hypoglycemia surveillance and population studies in diabetes patients

Detecting Hypoglycemia Incidents Reported in Patients\u27 Secure Messages: Using Cost-Sensitive Learning and Oversampling to Reduce Data Imbalance

BACKGROUND: Improper dosing of medications such as insulin can cause hypoglycemic episodes, which may lead to severe morbidity or even death. Although secure messaging was designed for exchanging nonurgent messages, patients sometimes report hypoglycemia events through secure messaging. Detecting these patient-reported adverse events may help alert clinical teams and enable early corrective actions to improve patient safety. OBJECTIVE: We aimed to develop a natural language processing system, called HypoDetect (Hypoglycemia Detector), to automatically identify hypoglycemia incidents reported in patients\u27 secure messages. METHODS: An expert in public health annotated 3000 secure message threads between patients with diabetes and US Department of Veterans Affairs clinical teams as containing patient-reported hypoglycemia incidents or not. A physician independently annotated 100 threads randomly selected from this dataset to determine interannotator agreement. We used this dataset to develop and evaluate HypoDetect. HypoDetect incorporates 3 machine learning algorithms widely used for text classification: linear support vector machines, random forest, and logistic regression. We explored different learning features, including new knowledge-driven features. Because only 114 (3.80%) messages were annotated as positive, we investigated cost-sensitive learning and oversampling methods to mitigate the challenge of imbalanced data. RESULTS: The interannotator agreement was Cohen kappa=.976. Using cross-validation, logistic regression with cost-sensitive learning achieved the best performance (area under the receiver operating characteristic curve=0.954, sensitivity=0.693, specificity 0.974, F1 score=0.590). Cost-sensitive learning and the ensembled synthetic minority oversampling technique improved the sensitivity of the baseline systems substantially (by 0.123 to 0.728 absolute gains). Our results show that a variety of features contributed to the best performance of HypoDetect. CONCLUSIONS: Despite the challenge of data imbalance, HypoDetect achieved promising results for the task of detecting hypoglycemia incidents from secure messages. The system has a great potential to facilitate early detection and treatment of hypoglycemia

Evaluating diverse electronic consultation programs with a common framework.

BackgroundElectronic consultation is an emerging mode of specialty care delivery that allows primary care providers and their patients to obtain specialist expertise without an in-person visit. While studies of individual programs have demonstrated benefits related to timely access to specialty care, electronic consultation programs have not achieved widespread use in the United States. The lack of common evaluation metrics across health systems and concerns related to the generalizability of existing evaluation efforts may be hampering further growth. We sought to identify gaps in knowledge related to the implementation of electronic consultation programs and develop a set of shared evaluation measures to promote further diffusion.MethodsUsing a case study approach, we apply the Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) and the Quadruple Aim frameworks of evaluation to examine electronic consultation implementation across diverse delivery systems. Data are from 4 early adopter healthcare delivery systems (San Francisco Health Network, Mayo Clinic, Veterans Administration, Champlain Local Health Integration Network) that represent varied organizational structures, care for different patient populations, and have well-established multi-specialty electronic consultation programs. Data sources include published and unpublished quantitative data from each electronic consultation database and qualitative data from systems' end-users.ResultsOrganizational drivers of electronic consultation implementation were similar across the systems (challenges with timely and/or efficient access to specialty care), though unique system-level facilitators and barriers influenced reach, adoption and design. Effectiveness of implementation was consistent, with improved patient access to timely, perceived high-quality specialty expertise with few negative consequences, garnering high satisfaction among end-users. Data about patient-specific clinical outcomes are lacking, as are policies that provide guidance on the legal implications of electronic consultation and ideal remuneration strategies.ConclusionA core set of effectiveness and implementation metrics rooted in the Quadruple Aim may promote data-driven improvements and further diffusion of successful electronic consultation programs

A Natural Language Processing System That Links Medical Terms in Electronic Health Record Notes to Lay Definitions: System Development Using Physician Reviews

BACKGROUND: Many health care systems now allow patients to access their electronic health record (EHR) notes online through patient portals. Medical jargon in EHR notes can confuse patients, which may interfere with potential benefits of patient access to EHR notes. OBJECTIVE: The aim of this study was to develop and evaluate the usability and content quality of NoteAid, a Web-based natural language processing system that links medical terms in EHR notes to lay definitions, that is, definitions easily understood by lay people. METHODS: NoteAid incorporates two core components: CoDeMed, a lexical resource of lay definitions for medical terms, and MedLink, a computational unit that links medical terms to lay definitions. We developed innovative computational methods, including an adapted distant supervision algorithm to prioritize medical terms important for EHR comprehension to facilitate the effort of building CoDeMed. Ten physician domain experts evaluated the user interface and content quality of NoteAid. The evaluation protocol included a cognitive walkthrough session and a postsession questionnaire. Physician feedback sessions were audio-recorded. We used standard content analysis methods to analyze qualitative data from these sessions. RESULTS: Physician feedback was mixed. Positive feedback on NoteAid included (1) Easy to use, (2) Good visual display, (3) Satisfactory system speed, and (4) Adequate lay definitions. Opportunities for improvement arising from evaluation sessions and feedback included (1) improving the display of definitions for partially matched terms, (2) including more medical terms in CoDeMed, (3) improving the handling of terms whose definitions vary depending on different contexts, and (4) standardizing the scope of definitions for medicines. On the basis of these results, we have improved NoteAid\u27s user interface and a number of definitions, and added 4502 more definitions in CoDeMed. CONCLUSIONS: Physician evaluation yielded useful feedback for content validation and refinement of this innovative tool that has the potential to improve patient EHR comprehension and experience using patient portals. Future ongoing work will develop algorithms to handle ambiguous medical terms and test and evaluate NoteAid with patients

Secure Messaging, Diabetes Self-management, and the Importance of Patient Autonomy: a Mixed Methods Study

BACKGROUND: Diabetes is a complex, chronic disease that requires patients\u27 effective self-management between clinical visits; this in turn relies on patient self-efficacy. The support of patient autonomy from healthcare providers is associated with better self-management and greater diabetes self-efficacy. Effective provider-patient secure messaging (SM) through patient portals may improve disease self-management and self-efficacy. SM that supports patients\u27 sense of autonomy may mediate this effect by providing patients ready access to their health information and better communication with their clinical teams. OBJECTIVE: We examined the association between healthcare team-initiated SM and diabetes self-management and self-efficacy, and whether this association was mediated by patients\u27 perceptions of autonomy support from their healthcare teams. DESIGN: We surveyed and analyzed content of messages sent to a sample of patients living with diabetes who use the SM feature on the VA\u27s My HealtheVet patient portal. PARTICIPANTS: Four hundred forty-six veterans with type 2 diabetes who were sustained users of SM. MAIN MEASURES: Proactive (healthcare team-initiated) SM (0 or \u3e /= 1 messages); perceived autonomy support; diabetes self-management; diabetes self-efficacy. KEY RESULTS: Patients who received at least one proactive SM from their clinical team were significantly more likely to engage in better diabetes self-management and report a higher sense of diabetes self-efficacy. This relationship was mediated by the patient\u27s perception of autonomy support. The majority of proactive SM discussed scheduling, referrals, or other administrative content. Patients\u27 responses to team-initiated communication promoted patient engagement in diabetes self-management behaviors. CONCLUSIONS: Perceived autonomy support is important for diabetes self-management and self-efficacy. Proactive communication from clinical teams to patients can help to foster a patient\u27s sense of autonomy and encourage better diabetes self-management and self-efficacy

Electronic consultations (E-consults) and their outcomes: a systematic review

OBJECTIVE: Electronic consultations (e-consults) are clinician-to-clinician communications that may obviate face-to-face specialist visits. E-consult programs have spread within the US and internationally despite limited data on outcomes. We conducted a systematic review of the recent peer-reviewed literature on the effect of e-consults on access, cost, quality, and patient and clinician experience and identified the gaps in existing research on these outcomes. MATERIALS AND METHODS: We searched 4 databases for empirical studies published between 1/1/2015 and 2/28/2019 that reported on one or more outcomes of interest. Two investigators reviewed titles and abstracts. One investigator abstracted information from each relevant article, and another confirmed the abstraction. We applied the GRADE criteria for the strength of evidence for each outcome. RESULTS: We found only modest empirical evidence for effectiveness of e-consults on important outcomes. Most studies are observational and within a single health care system, and comprehensive assessments are lacking. For those outcomes that have been reported, findings are generally positive, with mixed results for clinician experience. These findings reassure but also raise concern for publication bias. CONCLUSION: Despite stakeholder enthusiasm and encouraging results in the literature to date, more rigorous study designs applied across all outcomes are needed. Policy makers need to know what benefits may be expected in what contexts, so they can define appropriate measures of success and determine how to achieve them. Informatics Association 2019. This work is written by US Government employees and is in the public domain in the US

Impact of electronic consultation (e-consult) on timeliness and guideline concordance of workups leading to thyroid nodule fine-needle aspiration biopsy

OBJECTIVE: Electronic consultations (e-consults) are commonly used to obtain endocrinology input on clinical questions without a face-to-face visit, but there are sparse data on quality of care resulting from e-consults for specific conditions. We examined workups resulting in thyroid nodule fine-needle aspiration (FNA) biopsy to examine whether endocrinology e-consults were more timely and similarly guideline concordant as compared to endocrinology face-to-face visits, and whether endocrinology e-consults were more guideline concordant as compared to workups without endocrinology input. METHODS: We conducted a retrospective chart review of 302 thyroid FNA biopsies conducted in the Veterans Affairs (VA) health system between 5/1/2017 and 2/4/2020 (e-consult, N=99; face-to-face visit, N=100; no endocrinology input, N=103). We used t-tests to compare timeliness and chi-square tests to compare the proportion of guideline-concordant workups. We used multivariable linear and logistic models to control for demographic factors. RESULTS: FNAs preceded by endocrinology e-consult had more timely workups compared to those preceded by endocrinology face-to-face visits in terms of days elapsed between referral and FNA biopsy [geometric mean [95% confidence interval]: 44.7 [37.2, 53.7] days vs. 61.7 [52.5, 72.4] days, t(195)=2.55, P=0.01]. The difference in the summary measure of guideline concordance across groups was not statistically significant (P=0.38). CONCLUSION: E-consults were faster than face-to-face consults and similar guideline concordant as compared to both face-to-face consults and no endocrinology input for workups resulting in FNA. Decisions about appropriate use of e-consults for thyroid nodules should account for these data while also considering potential benefits of direct patient-endocrinologist interaction for complex situations

Patients’ Experience of Specialty Care Coordination: Survey Development and Validation

Purpose: Specialty care coordination relies on information flowing bidirectionally between all three participants in the “specialty care triad” — patients, primary care providers (PCPs), and specialists. Measures of coordination should strive to account for the perspectives of each. As we previously developed two surveys to measure coordination of specialty care as experienced by PCPs and specialists, this study aimed to develop and evaluate the psychometric properties of a related survey of specialty care coordination as experienced by the patient, thereby completing the suite of surveys among the triad. Methods: We developed a draft survey based on literature review, patient interviews, adaptation of existing measures, and development of new items. Survey responses were collected via mail and online in two waves, August 2019–November 2019 and September 2020–May 2021, among patients (N = 939) receiving medical specialty care and primary care in the Veterans Affairs health system. Exploratory and confirmatory factor analysis were used to assess scale structure. Multiple linear regression was used to examine the relationship of the final coordination scales to patients’ overall experience of specialty care coordination. Results: A 38-item measure representing 10 factors that assess the patient’s experience of coordination in specialty care among the patient, PCP, and specialist was finalized. Scales demonstrated good internal consistency reliability and, together, explained 59% of the variance in overall coordination. Analyses revealed an unexpected construct describing organization of care between patient and specialist that accounted for patient goals and preferences; this 10-item scale was named Patient-Centered Care Coordination. Conclusions: The final survey, Coordination of Specialty Care – Patient, or CSC-Patient for short, is a reliable instrument that can be used alone or with its companions (CSC-PCP, CSC-Specialist) to provide a detailed assessment of specialty care coordination and identify targets for coordination improvement