Postmarket sequential database surveillance of medical products

Thesis (Ph. D.)--Massachusetts Institute of Technology, Engineering Systems Division, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 193-212).This dissertation focuses on the capabilities of a novel public health data system - the Sentinel System - to supplement existing postmarket surveillance systems of the U.S. Food and Drug Administration (FDA). The Sentinel System is designed to identify and assess safety risks associated with drugs, therapeutic biologics, vaccines, and medical devices that emerge post-licensure. Per the initiating legislation, the FDA must complete a priori evaluations of the Sentinel System's technical capabilities to support regulatory decision-making. This research develops qualitative and quantitative tools to aid the FDA in such evaluations, particularly with regard to the Sentinel System's novel sequential database surveillance capabilities. Sequential database surveillance is a "near real-time" sequential statistical method to evaluate pre-specified exposure-outcome pairs. A "signal" is detected when the data suggest an excess risk that is statistically significant. The qualitative tool - the Sentinel System Pre- Screening Checklist - is designed to determine whether the Sentinel System is well suited, on its face, to evaluate a pre-specified exposure-outcome pair. The quantitative tool - the Sequential Database Surveillance Simulator - allows the user to explore virtually whether sequential database surveillance of a particular exposure-outcome pair is likely to generate evidence to identify and assess safety risks in a timely manner to support regulatory decision-making. Particular attention is paid to accounting for uncertainties including medical product adoption and utilization, misclassification error, and the unknown true excess risk in the environment. Using vaccine examples and the simulator to illustrate, this dissertation first demonstrates the tradeoffs associated with sample size calculations in sequential statistical analysis, particularly the tradeoff between statistical power and median sample size. Second, it demonstrates differences in performance between various surveillance configurations when using distributed database systems. Third, it demonstrates the effects of misclassification error on sequential database surveillance, and specifically how such errors may be accounted for in the design of surveillance. Fourth, it considers the complexities of modeling new medical product adoption, and specifically, the existence of a "dual market" phenomenon for these new medical products. This finding raises non-trivial generalizability concerns regarding evidence generated via sequential database surveillance when performed immediately post-licensure.by Judith C. Maro.Ph.D

Development of a public health information infrastructure for postmarket evidence

Thesis (S.M. in Technology and Policy)--Massachusetts Institute of Technology, Engineering Systems Division, Technology and Policy Program, 2009.Includes bibliographical references (p. 133-153).Postmarket data on prescription medical product performance has historically been incomplete, underutilized, and mismanaged to inform safety and comparative clinical effectiveness. Congress has tasked the Food and Drug Administration to build a public health information infrastructure for drug safety. It also has allotted $1.1 billion dollars in new spending for comparative effectiveness research. A singular, shared, multi-purpose public health information infrastructure can be built to serve both these needs and others. It can be used by multiple public health agencies under a coordinating framework. A new independent public health authority is best positioned to manage that framework and to negotiate the security, legal, proprietary, and privacy barriers that accompany requests to access large amounts of patient data. Such a design protects privacy, avoids duplication, leverages investment, and promotes sustainability in what is truly a "greenfield" opportunity in the United States. Consequently, the policy window to influence the system design is now. Personal health data is the scarce resource needed to constitute this infrastructure. Citizens have a right and responsibility to re-examine how postmarket data is used to measure safety and comparative clinical effectiveness. A public process to establish new classification schemes that set benefit-risk targets for classes of prescription medical products is needed. Such schemes would differentiate products according to therapeutic need, expected length of treatment, expected patient population, novelty of treatment, and availability of substitutes.(cont.) These classes would prompt different postmarket requirements according the needs and values of the affected patient population. Data collection, data analysis, risk management strategies, and reimbursement strategies would logically follow from this classification. In this paper, inadequate historical postmarket data generation mechanisms and risk management plans are reviewed. Specific attention is given to the failed use of "carrots" and "sticks" to elicit desired behavior. Next, an analysis of stakeholder interests and desired public health outcomes is performed. Policy goals for a public health information infrastructure are outlined along with strategies to achieve those goals.by Judith C. Maro.S.M.in Technology and Polic

Responding to Vaccine Safety Signals during Pandemic Influenza: A Modeling Study

Background: Managing emerging vaccine safety signals during an influenza pandemic is challenging. Federal regulators must balance vaccine risks against benefits while maintaining public confidence in the public health system. Methods: We developed a multi-criteria decision analysis model to explore regulatory decision-making in the context of emerging vaccine safety signals during a pandemic. We simulated vaccine safety surveillance system capabilities and used an age-structured compartmental model to develop potential pandemic scenarios. We used an expert-derived multi-attribute utility function to evaluate potential regulatory responses by combining four outcome measures into a single measure of interest: 1) expected vaccination benefit from averted influenza; 2) expected vaccination risk from vaccine-associated febrile seizures; 3) expected vaccination risk from vaccine-associated Guillain-Barre Syndrome; and 4) expected change in vaccine-seeking behavior in future influenza seasons. Results: Over multiple scenarios, risk communication, with or without suspension of vaccination of high-risk persons, were the consistently preferred regulatory responses over no action or general suspension when safety signals were detected during a pandemic influenza. On average, the expert panel valued near-term vaccine-related outcomes relative to long-term projected outcomes by 3∶1. However, when decision-makers had minimal ability to influence near-term outcomes, the response was selected primarily by projected impacts on future vaccine-seeking behavior. Conclusions: The selected regulatory response depends on how quickly a vaccine safety signal is identified relative to the peak of the pandemic and the initiation of vaccination. Our analysis suggested two areas for future investment: efforts to improve the size and timeliness of the surveillance system and behavioral research to understand changes in vaccine-seeking behavior

A Synthesis of Current Surveillance Planning Methods for the Sequential Monitoring of Drug and Vaccine Adverse Effects Using Electronic Health Care Data

Introduction: The large-scale assembly of electronic health care data combined with the use of sequential monitoring has made proactive postmarket drug- and vaccine-safety surveillance possible. Although sequential designs have been used extensively in randomized trials, less attention has been given to methods for applying them in observational electronic health care database settings. Existing Methods: We review current sequential-surveillance planning methods from randomized trials, and the Vaccine Safety Datalink (VSD) and Mini-Sentinel Pilot projects—two national observational electronic health care database safety monitoring programs. Future Surveillance Planning: Based on this examination, we suggest three steps for future surveillance planning in health care databases: (1) prespecify the sequential design and analysis plan, using available feasibility data to reduce assumptions and minimize later changes to initial plans; (2) assess existing drug or vaccine uptake, to determine if there is adequate information to proceed with surveillance, before conducting more resource-intensive planning; and (3) statistically evaluate and clearly communicate the sequential design with all those designing and interpreting the safety-surveillance results prior to implementation. Plans should also be flexible enough to accommodate dynamic and often unpredictable changes to the database information made by the health plans for administrative purposes. Conclusions: This paper is intended to encourage dialogue about establishing a more systematic, scalable, and transparent sequential design-planning process for medical-product safety-surveillance systems utilizing observational electronic health care databases. Creating such a framework could yield improvements over existing practices, such as designs with increased power to assess serious adverse events

Statistical Power for Postlicensure Medical Product Safety Data Mining

Objective: To perform sample size calculations when using tree-based scan statistics in longitudinal observational databases. Methods: Tree-based scan statistics enable data mining on epidemiologic datasets where thousands of disease outcomes are organized into hierarchical tree structures with automatic adjustment for multiple testing. We show how to evaluate the statistical power of the unconditional and conditional Poisson versions. The null hypothesis is that there is no increase in the risk for any of the outcomes. The alternative is that one or more outcomes have an excess risk. We varied the excess risk, total sample size, frequency of the underlying event rate, and the level of across-the-board health care utilization. We also quantified the reduction in statistical power resulting from specifying a risk window that was too long or too short. Results: For 500,000 exposed people, we had at least 98 percent power to detect an excess risk of 1 event per 10,000 exposed for all outcomes. In the presence of potential temporal confounding due to across-the-board elevations of health care utilization in the risk window, the conditional tree-based scan statistic controlled type I error well, while the unconditional version did not. Discussion: Data mining analyses using tree-based scan statistics expand the pharmacovigilance toolbox, ensuring adequate monitoring of thousands of outcomes of interest while controlling for multiple hypothesis testing. These power evaluations enable investigators to design and optimize implementation of retrospective data mining analyses