Corresponding sections and subsections among test corpora.

Corresponding sections and subsections among test corpora.</p

The overview of assessing heterogeneity of clinical sections across three electronic health records using embedding-based machine learning approaches.

EHR = Electronic health records; HL7-CDA = Health Level 7—Clinical Document Architecture; ML = Machine Learning; RF = Random Forest; BERT = Bidirectional Encoder Representations from Transformers; GEC = General Electronic Centricity EHR.</p

Differentiation of corresponding sections and subsections among corpora.

aOne clinical document standard of IC Chart Electronic Health Record (EHR) consists of sections ‘S,’ ‘O’, ‘A,’ and ‘P.’; The “Impression and Plan” section of the Cerner EHR, corresponds to “the Assessment and Plan” section of the General Electronic Centricity (GEC) EHR, which contains information on the “Problem,” “Medications,” and other sections; “Chief Complaint and Reason for Visit” sections of the Epic EHR are similar to “Problem” sections of the GEC EHR. (DOCX)</p

The algorithm of section classifiers.

BackgroundThe incorporation of information from clinical narratives is critical for computational phenotyping. The accurate interpretation of clinical terms highly depends on their associated context, especially the corresponding clinical section information. However, the heterogeneity across different Electronic Health Record (EHR) systems poses challenges in utilizing the section information.ObjectivesLeveraging the eMERGE heart failure (HF) phenotyping algorithm, we assessed the heterogeneity quantitatively through the performance comparison of machine learning (ML) classifiers which map clinical sections containing HF-relevant terms across different EHR systems to standard sections in Health Level 7 (HL7) Clinical Document Architecture (CDA).MethodsWe experimented with both random forest models with sentence-embedding features and bidirectional encoder representations from transformers models. We trained MLs using an automated labeled corpus from an EHR system that adopted HL7 CDA standard. We assessed the performance using a blind test set (n = 300) from the same EHR system and a gold standard (n = 900) manually annotated from three other EHR systems.ResultsThe F-measure of those ML models varied widely (0.00–0.91%), indicating MLs with one tuning parameter set were insufficient to capture sections across different EHR systems. The error analysis indicates that the section does not always comply with the corresponding standardized sections, leading to low performance.ConclusionsWe presented the potential use of ML techniques to map the sections containing HF-relevant terms in multiple EHR systems to standard sections. However, the findings suggested that the quality and heterogeneity of section structure across different EHRs affect applications due to the poor adoption of documentation standards.</div

Case^a and non-case^b for train set in GEC and the heart failure relevant clinical section in test sets.

Casea and non-caseb for train set in GEC and the heart failure relevant clinical section in test sets.</p

The distribution of the sections and subsections in the test.

The distribution of the sections and subsections in the test.</p

Evaluation of binary embedding-based ML models regarding the section.

Evaluation of binary embedding-based ML models regarding the section.</p