Pharmacoepidemiol Drug Saf
- Publication date
- Publisher
Abstract
Purpose:The objective was to develop a natural language processing (NLP)
algorithm to identify vaccine-related anaphylaxis from plain-text clinical
notes, and to implement the algorithm at five health care systems in the
Vaccine Safety Datalink.Methods:The NLP algorithm was developed using an internal NLP tool and
training dataset of 311 potential anaphylaxis cases from Kaiser Permanente
Southern California (KPSC). We applied the algorithm to the notes of another
731 potential cases (423 from KPSC; 308 from other sites) with relevant
codes (ICD-9-CM diagnosis codes for anaphylaxis, vaccine adverse reactions,
and allergic reactions; Healthcare Common Procedure Coding System codes for
epinephrine administration). NLP results were compared against a reference
standard of chart reviewed and adjudicated cases. The algorithm was then
separately applied to the notes of 6 427 359 KPSC vaccination visits (9 402
194 vaccine doses) without relevant codes.Results:At KPSC, NLP identified 12 of 16 true vaccine-related cases and
achieved a sensitivity of 75.0%, specificity of 98.5%, positive predictive
value (PPV) of 66.7%, and negative predictive value of 99.0% when applied to
notes of patients with relevant diagnosis codes. NLP did not identify the
five true cases at other sites. When NLP was applied to the notes of KPSC
patients without relevant codes, it captured eight additional true cases
confirmed by chart review and adjudication.Conclusions:The current study demonstrated the potential to apply rule-based NLP
algorithms to clinical notes to identify anaphylaxis cases. Increasing the
size of training data, including clinical notes from all participating study
sites in the training data, and preprocessing the clinical notes to handle
special characters could improve the performance of the NLP algorithms. We
recommend adding an NLP process followed by manual chart review in future
vaccine safety studies to improve sensitivity and efficiency.CC999999/ImCDC/Intramural CDC HHS/United StatesCC/CDC HHS/United States2021-02-01T00:00:00Z31797475PMC75288878412vault:3605