Validation of Acute Myocardial Infarction (AMI) in the FDA’s Mini-Sentinel Distributed Database

The Food and Drug Administration’s (FDA) Mini-Sentinel is a pilot program that aims to conduct active surveillance to detect and refine safety signals that emerge for marketed medical products. The purpose of this Mini-Sentinel AMI Validation project was to: (a) develop and design an abstraction and adjudication process to use when full text medical record review is required to confirm a coded diagnosis; and (b) to test this approach by validating a code algorithm for acute myocardial infarction (AMI)

Design for validation of acute myocardial infarction cases in Mini-Sentinel

PURPOSE: To describe the acute myocardial infarction (AMI) validation project, a test case for health outcome validation within the US Food and Drug Administration-funded Mini-Sentinel pilot program. METHODS: The project consisted of four parts: (i) case identification-developing an algorithm based on the International Classification of Diseases, Ninth Revision, to identify hospitalized AMI patients within the Mini-Sentinel Distributed Database; (ii) chart retrieval-establishing procedures that ensured patient privacy (collection and transfer of minimum necessary amount of information, and redaction of direct identifiers to validate potential cases of AMI); (iii) abstraction and adjudication-trained nurse abstractors gathered key data using a standardized form with cardiologist adjudication; and (iv) calculation of the positive predictive value of the constructed algorithm. RESULTS: Key decision points included (i) breadth of the AMI algorithm, (ii) centralized versus distributed abstraction, and (iii) approaches to maintaining patient privacy and to obtaining charts for public health purposes. We used an algorithm limited to International Classification of Diseases, Ninth Revision, codes 410.x0-410.x1. Centralized data abstraction was performed because of the modest number of charts requested ( CONCLUSIONS: We have established a process to validate AMI within Mini-Sentinel, which may be used for other health outcomes. Challenges include the following: (i) ensuring that only minimum necessary data are transmitted by Data Partners for centralized chart review, (ii) establishing procedures to maintain data privacy while still allowing for timely access to medical charts, and (iii) securing access to charts for public health uses that do not require approval from an institutional review board while maintaining patient privacy

Validation of anaphylaxis in the Food and Drug Administration\u27s Mini-Sentinel

PURPOSE: We aim to develop and validate the positive predictive value (PPV) of an algorithm to identify anaphylaxis using health plan administrative and claims data. Previously published PPVs for anaphylaxis using International Classification of Diseases, ninth revision, Clinical Modification (ICD-9-CM) codes range from 52% to 57%. METHODS: We conducted a retrospective study using administrative and claims data from eight health plans. Using diagnosis and procedure codes, we developed an algorithm to identify potential cases of anaphylaxis from the Mini-Sentinel Distributed Database between January 2009 and December 2010. A random sample of medical charts (n = 150) was identified for chart abstraction. Two physician adjudicators reviewed each potential case. Using physician adjudicator judgments on whether the case met diagnostic criteria for anaphylaxis, we calculated a PPV for the algorithm. RESULTS: Of the 122 patients for whom complete charts were received, 77 were judged by physician adjudicators to have anaphylaxis. The PPV for the algorithm was 63.1% (95%CI: 53.9-71.7%), using the clinical criteria by Sampson as the gold standard. The PPV was highest for inpatient encounters with ICD-9-CM codes of 995.0 or 999.4. By combining only the top performing ICD-9-CM codes, we identified an algorithm with a PPV of 75.0%, but only 66% of cases of anaphylaxis were identified using this modified algorithm. CONCLUSIONS: The PPV for the ICD-9-CM-based algorithm for anaphylaxis was slightly higher than PPV estimates reported in prior studies, but remained low. We were able to identify an algorithm that optimized the PPV but demonstrated lower sensitivity for anaphylactic events

Validation of acute myocardial infarction in the Food and Drug Administration\u27s Mini-Sentinel program

PURPOSE: To validate an algorithm based upon International Classification of Diseases, 9(th) revision, Clinical Modification (ICD-9-CM) codes for acute myocardial infarction (AMI) documented within the Mini-Sentinel Distributed Database (MSDD). METHODS: Using an ICD-9-CM-based algorithm (hospitalized patients with 410.x0 or 410.x1 in primary position), we identified a random sample of potential cases of AMI in 2009 from four Data Partners participating in the Mini-Sentinel Program. Cardiologist reviewers used information abstracted from hospital records to assess the likelihood of an AMI diagnosis based on criteria from the Joint European Society of Cardiology and American College of Cardiology Global Task Force. Positive predictive values (PPVs) of the ICD-9-based algorithm were calculated. RESULTS: Of the 153 potential cases of AMI identified, hospital records for 143 (93%) were retrieved and abstracted. Overall, the PPV was 86.0% (95% confidence interval; 79.2%, 91.2%). PPVs ranged from 76.3% to 94.3% across the four Data Partners. CONCLUSIONS: The overall PPV of potential AMI cases, as identified using an ICD-9-CM-based algorithm, may be acceptable for safety surveillance; however, PPVs do vary across Data Partners. This validation effort provides a contemporary estimate of the reliability of this algorithm for use in future surveillance efforts conducted using the Food and Drug Administration\u27s MSDD. Copyright © 2012 John Wiley & Sons, Ltd

Validation of Claims-Based Algorithm for Lyme Disease, Massachusetts, USA

Compared with notifiable disease surveillance, claims-based algorithms estimate higher Lyme disease incidence, but their accuracy is unknown. We applied a previously developed Lyme disease algorithm (diagnosis code plus antimicrobial drug prescription dispensing within 30 days) to an administrative claims database in Massachusetts, USA, to identify a Lyme disease cohort during July 2000–June 2019. Clinicians reviewed and adjudicated medical charts from a cohort subset by using national surveillance case definitions. We calculated positive predictive values (PPVs). We identified 12,229 Lyme disease episodes in the claims database and reviewed and adjudicated 128 medical charts. The algorithmʼs PPV for confirmed, probable, or suspected cases was 93.8% (95% CI 88.1%–97.3%); the PPV was 66.4% (95% CI 57.5%–74.5%) for confirmed and probable cases only. In a high incidence setting, a claims-based algorithm identified cases with a high PPV, suggesting it can be used to assess Lyme disease burden and supplement traditional surveillance data