22 research outputs found
Multimodal Machine Learning for Automated ICD Coding
This study presents a multimodal machine learning model to predict ICD-10
diagnostic codes. We developed separate machine learning models that can handle
data from different modalities, including unstructured text, semi-structured
text and structured tabular data. We further employed an ensemble method to
integrate all modality-specific models to generate ICD-10 codes. Key evidence
was also extracted to make our prediction more convincing and explainable. We
used the Medical Information Mart for Intensive Care III (MIMIC -III) dataset
to validate our approach. For ICD code prediction, our best-performing model
(micro-F1 = 0.7633, micro-AUC = 0.9541) significantly outperforms other
baseline models including TF-IDF (micro-F1 = 0.6721, micro-AUC = 0.7879) and
Text-CNN model (micro-F1 = 0.6569, micro-AUC = 0.9235). For interpretability,
our approach achieves a Jaccard Similarity Coefficient (JSC) of 0.1806 on text
data and 0.3105 on tabular data, where well-trained physicians achieve 0.2780
and 0.5002 respectively.Comment: Machine Learning for Healthcare 201
MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning
Learning multimodal representations involves integrating information from
multiple heterogeneous sources of data. In order to accelerate progress towards
understudied modalities and tasks while ensuring real-world robustness, we
release MultiZoo, a public toolkit consisting of standardized implementations
of > 20 core multimodal algorithms and MultiBench, a large-scale benchmark
spanning 15 datasets, 10 modalities, 20 prediction tasks, and 6 research areas.
Together, these provide an automated end-to-end machine learning pipeline that
simplifies and standardizes data loading, experimental setup, and model
evaluation. To enable holistic evaluation, we offer a comprehensive methodology
to assess (1) generalization, (2) time and space complexity, and (3) modality
robustness. MultiBench paves the way towards a better understanding of the
capabilities and limitations of multimodal models, while ensuring ease of use,
accessibility, and reproducibility. Our toolkits are publicly available, will
be regularly updated, and welcome inputs from the community.Comment: JMLR Open Source Software 2023, Code available at
https://github.com/pliang279/MultiBenc
Automated Clinical Coding:What, Why, and Where We Are?
Clinical coding is the task of transforming medical information in a
patient's health records into structured codes so that they can be used for
statistical analysis. This is a cognitive and time-consuming task that follows
a standard process in order to achieve a high level of consistency. Clinical
coding could potentially be supported by an automated system to improve the
efficiency and accuracy of the process. We introduce the idea of automated
clinical coding and summarise its challenges from the perspective of Artificial
Intelligence (AI) and Natural Language Processing (NLP), based on the
literature, our project experience over the past two and half years (late 2019
- early 2022), and discussions with clinical coding experts in Scotland and the
UK. Our research reveals the gaps between the current deep learning-based
approach applied to clinical coding and the need for explainability and
consistency in real-world practice. Knowledge-based methods that represent and
reason the standard, explainable process of a task may need to be incorporated
into deep learning-based methods for clinical coding. Automated clinical coding
is a promising task for AI, despite the technical and organisational
challenges. Coders are needed to be involved in the development process. There
is much to achieve to develop and deploy an AI-based automated system to
support coding in the next five years and beyond.Comment: accepted for npj Digital Medicin
Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing
Strategy training is a multidisciplinary rehabilitation approach that teaches
skills to reduce disability among those with cognitive impairments following a
stroke. Strategy training has been shown in randomized, controlled clinical
trials to be a more feasible and efficacious intervention for promoting
independence than traditional rehabilitation approaches. A standardized
fidelity assessment is used to measure adherence to treatment principles by
examining guided and directed verbal cues in video recordings of rehabilitation
sessions. Although the fidelity assessment for detecting guided and directed
verbal cues is valid and feasible for single-site studies, it can become labor
intensive, time consuming, and expensive in large, multi-site pragmatic trials.
To address this challenge to widespread strategy training implementation, we
leveraged natural language processing (NLP) techniques to automate the strategy
training fidelity assessment, i.e., to automatically identify guided and
directed verbal cues from video recordings of rehabilitation sessions. We
developed a rule-based NLP algorithm, a long-short term memory (LSTM) model,
and a bidirectional encoder representation from transformers (BERT) model for
this task. The best performance was achieved by the BERT model with a 0.8075
F1-score. This BERT model was verified on an external validation dataset
collected from a separate major regional health system and achieved an F1 score
of 0.8259, which shows that the BERT model generalizes well. The findings from
this study hold widespread promise in psychology and rehabilitation
intervention research and practice.Comment: Accepted at the AMIA Informatics Summit 202
ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Network
Automated ICD coding, which assigns the International Classification of
Disease codes to patient visits, has attracted much research attention since it
can save time and labor for billing. The previous state-of-the-art model
utilized one convolutional layer to build document representations for
predicting ICD codes. However, the lengths and grammar of text fragments, which
are closely related to ICD coding, vary a lot in different documents.
Therefore, a flat and fixed-length convolutional architecture may not be
capable of learning good document representations. In this paper, we proposed a
Multi-Filter Residual Convolutional Neural Network (MultiResCNN) for ICD
coding. The innovations of our model are two-folds: it utilizes a multi-filter
convolutional layer to capture various text patterns with different lengths and
a residual convolutional layer to enlarge the receptive field. We evaluated the
effectiveness of our model on the widely-used MIMIC dataset. On the full code
set of MIMIC-III, our model outperformed the state-of-the-art model in 4 out of
6 evaluation metrics. On the top-50 code set of MIMIC-III and the full code set
of MIMIC-II, our model outperformed all the existing and state-of-the-art
models in all evaluation metrics. The code is available at
https://github.com/foxlf823/Multi-Filter-Residual-Convolutional-Neural-Network
Predicting Multiple ICD-10 Codes from Brazilian-Portuguese Clinical Notes
ICD coding from electronic clinical records is a manual, time-consuming and
expensive process. Code assignment is, however, an important task for billing
purposes and database organization. While many works have studied the problem
of automated ICD coding from free text using machine learning techniques, most
use records in the English language, especially from the MIMIC-III public
dataset. This work presents results for a dataset with Brazilian Portuguese
clinical notes. We develop and optimize a Logistic Regression model, a
Convolutional Neural Network (CNN), a Gated Recurrent Unit Neural Network and a
CNN with Attention (CNN-Att) for prediction of diagnosis ICD codes. We also
report our results for the MIMIC-III dataset, which outperform previous work
among models of the same families, as well as the state of the art. Compared to
MIMIC-III, the Brazilian Portuguese dataset contains far fewer words per
document, when only discharge summaries are used. We experiment concatenating
additional documents available in this dataset, achieving a great boost in
performance. The CNN-Att model achieves the best results on both datasets, with
micro-averaged F1 score of 0.537 on MIMIC-III and 0.485 on our dataset with
additional documents.Comment: Accepted at BRACIS 202