Development and external validation of automated ICD-10 coding from discharge summaries using deep learning approaches

Objectives: To develop an automated international classification of diseases (ICD) coding tool using natural language processing (NLP) and discharge summary texts from Thailand. Materials and methods: The development phase included 15,329 discharge summaries from Ramathibodi Hospital from January 2015 to December 2020. The external validation phase included Medical Information Mart for Intensive Care III (MIMIC-III) data. Three algorithms were developed: naïve Bayes with term frequency-inverse document frequency (NB-TF-IDF), convolutional neural network with neural word embedding (CNN-NWE), and CNN with PubMedBERT (CNN-PubMedBERT). In addition, two state-of-the-art models were also considered; convolutional attention for multi-label classification (CAML) and pretrained language models for automatic ICD coding (PLM-ICD). Results: The CNN-PubMedBERT model provided average micro- and macro-area under precision-recall curve (AUPRC) of 0.6605 and 0.5538, which outperformed CNN-NWE (0.6528 and 0.5564), NB-TF-IDF (0.4441 and 0.3562), and CAML (0.6257 and 0.4964), with corresponding differences of (0.0077 and −0.0026), (0.2164 and 0.1976), and (0.0348 and 0.0574), respectively. However, CNN-PubMedBERT performed less well relative to PLM-ICD, with corresponding AUPRCs of 0.7202 and 0.5865. The CNN-PubMedBERT model was externally validated using two subsets of MIMIC-III; MIMIC-ICD-10, and MIMIC-ICD-9 datasets, which contained 40,923 and 31,196 discharge summaries. The average micro-AUPRCs were 0.3745, 0.6878, and 0.6699, corresponding to directly predictive MIMIC-ICD-10, MIMIC-ICD-10 fine-tuning, and MIMIC-ICD-9 fine-tuning approaches; the average macro-AUPRCs for the corresponding models were 0.2819, 0.4219 and 0.5377, respectively. Discussion: CNN-PubMedBERT performed second-best to PLM-ICD, with considerable variation observed between average micro- and macro-AUPRC, especially for external validation, generally indicating good overall prediction but limited predictive value for small sample sizes. External validation in a US cohort demonstrated a higher level of model prediction performance. Conclusion: Both PLM-ICD and CNN-PubMedBERT models may provide useful tools for automated ICD-10 coding. Nevertheless, further evaluation and validation within Thai and Asian healthcare systems may prove more informative for clinical application

Systematic review of natural language processing for recurrent cancer detection from electronic medical records

This systematic review was conducted to explore natural language processing (NLP) focusing on text representation techniques and algorithms used previously to identify recurrent cancer diagnoses from electronic medical records (EMR), and an assessment of their detection performance. Relevant studies were identified from PubMed, Scopus, ACM Digital Library, and IEEE databases since inception to August 18, 2022. Data, including text representation methods, model algorithms and performance, and type of clinical notes, were extracted from individual studies by two independent reviewers. Study risk of bias was assessed using the prediction model risk of bias assessment tool. Of the 412 studies identified, 17 were eligible for inclusion, with 15 representing models that were not externally validated. Three text representations were used: statistical, context-free, and contextual representations (bidirectional encoder representations from transformers (BERT) and its variants), from 12, 6, and 3 studies, respectively. The corresponding median harmonic precision and recall means (F1 scores) for these representations were 0.43, 0.87, and 0.72, respectively. The algorithms applied included rule-based, machine learning, and deep learning approaches with median F1 scores of 0.71, 0.43, and 0.76, respectively. In conclusion, this systematic review suggests that deep learning models that use PubMedBERT as a text representation perform best. These findings are clinically informative for the selection of appropriate approaches for the detection of recurrent cancer from electronic medical records

Informatics education in low-resource settings

Developing countries have the burden of acute and chronic diseases with the greatest health disparities. There is also a shortfall of more than four million healthcare workers worldwide, and the proportion is higher in less economically viable countries where the lack of proper trained healthcare workers is also compromised by the migration and departure of skilled personnel together with a frail infrastructure and a shortage of resources that cannot provide a proper scenario for an adequate healthcare system that will fulfill the population needs. The need for both technology infrastructure and individuals who have the skills to develop these systems is great, but so are the challenges in developing the needed workforce who are well-trained in informatics. This chapter describes the current informatics education efforts in three regions: Latin America, Sub-Saharan Africa and the Asia-Pacific region. The description of specific healthcare informatics education programs, the educational methods used and the challenges encountered are explored

Cartoon versus traditional self-study handouts for medical students: CARTOON randomized controlled trial

<p><b>Objective:</b> The objective of this study is to compare the effectiveness of a “cartoon-style” handout with a “traditional-style” handout in a self-study assignment for preclinical medical students.</p> <p><b>Methods:</b> Third-year medical students (<i>n</i><b> </b>=<b> </b>93) at the Faculty of Medicine Ramathibodi Hospital, Mahidol University, took a pre-learning assessment of their knowledge of intercostal chest drainage. They were then randomly allocated to receive either a “cartoon-style” or a “traditional-style” handout on the same topic. After studying these over a 2-week period, students completed a post-learning assessment and estimated their levels of reading completion.</p> <p><b>Results:</b> Of the 79 participants completing the post-learning test, those in the cartoon-style group achieved a score 13.8% higher than the traditional-style group (<i>p</i><b> </b>=<b> </b>0.018). A higher proportion of students in the cartoon-style group reported reading ≥75% of the handout content (70.7% versus 42.1%). In post-hoc analyses, students whose cumulative grade point averages (GPA) from previous academic assessments were in the middle and lower range achieved higher scores with the cartoon-style handout than with the traditional one. In the lower-GPA group, the use of a cartoon-style handout was independently associated with a higher score.</p> <p><b>Conclusions:</b> Students given a cartoon-style handout reported reading more of the material and achieved higher post-learning test scores than students given a traditional handout.</p