Exploring the impact of design criteria for reference sets on performance evaluation of signal detection algorithms: the case of drug-drug interactions.

PurposeTo evaluate the impact of multiple design criteria for reference sets that are used to quantitatively assess the performance of pharmacovigilance signal detection algorithms (SDAs) for drug-drug interactions (DDIs).MethodsStarting from a large and diversified reference set for two-way DDIs, we generated custom-made reference sets of various sizes considering multiple design criteria (e.g., adverse event background prevalence). We assessed differences observed in the performance metrics of three different SDAs when applied to FDA Adverse Event Reporting System (FAERS) data.ResultsFor some design criteria, the impact on the performance metrics was neglectable for the different SDAs (e.g., theoretical evidence associated with positive controls), while others (e.g., restriction to designated medical events, event background prevalence) seemed to have opposing and effects of different sizes on AUC and PPV estimates.ConclusionsThe relative composition of reference sets can significantly impact the evaluation metrics, potentially altering the conclusions regarding which methodologies are perceived to perform best. We therefore need to carefully consider the selection of controls to avoid misinterpretation of signals triggered by confounding factors rather than true associations as well as adding biases to our evaluation by "favouring" some algorithms while penalising others. This article is protected by copyright. All rights reserved

Towards the interoperability of spontaneous reporting systems in pharmacovigilance : a maturity model approach with a sociotechnical system focus

Thesis (PhD)--Stellenbosch University, 2020.ENGLISH ABSTRACT: It is universally accepted that all medicines have the potential to cause adverse drug reactions (ADRs) during the course of their normal therapeutic use. Drug safety surveillance during the post-marketing authorisation phase generates the majority of drug safety data, even more so than the clinical trials during the drug development process. Pharmacovigilance (PV), is based on the medical assessment of ADRs or drug-related problems, collected within organised health programmes. PV systems, by nature, are complex. The large number, fragmentation, and complexity of existing PV systems, the equally large number of stakeholders within such systems (i.e. pharmaceutical companies, government regulatory authorities, national and international clinical regulatory bodies, healthcare workers, etc.), as well as the significant number of dimensions along which the effectiveness and efficiency could be in uenced and also measured, adds to this complexity. The primary goal of any PV system is to improve and protect patient safety by enabling health care professionals to make more informed therapeutic decisions. Achieving this goal is dependent on the successful communication of relevant ADR information from the patient to the relevant PV authority. One such method of communication is the spontaneous reporting of ADRs, which is widely regarded as the cornerstone of data generation in PV during post-marketing authorisation safety surveillance. Currently, spontaneous reporting systems (SRSs) are faced with problems such as under-reporting and the communication of incomplete, unrepresentative, and uncontrolled data. The lack of standardisation and interoperability among these systems results in a reduced capability to detect and characterise new adverse drug interactions and ADRs. The primary obstacle to achieving interoperability between SRSs is the fundamental difference in the purpose of the existing SRSs. Stakeholders in the PV system operate SRSs with different goals and perspectives such as maintaining regulatory compliance, mitigating financial risk, and for the protection and promotion of patient safety in public health programmes. The aim of this study is to contribute towards the interoperability of SRSs in the PV landscape through the development of a novel maturity model with a sociotechnical system focus. The aim of the model is to promote and improve interoperability by addressing the degree of integration of systems involved, provide guidance on which system components need to be improved, as well as provide a means for measuring interoperability progress across the community of SRSs in the global PV landscape. A multidisciplinary literature review covering PV, capability maturity models, interoperability, and sociotechnical systems served as a theoretical foundation for the development of the model. The development of the model followed an adaptation of the 8-phase procedural model for developing maturity models, proposed by Becker et al. (2009). A comparison of 18 existing maturity models in the fields of: (i) PV; (ii) eHealth; (iii) eHealth/interoperability; (iv) interoperability; and (v) IT infrastructure, was conducted. The model is made up of three domains, seven subdomains, and thirty dimensions which were identified as a result of the preceding literature review and comparison of existing models. Through a combination of verification and validation processes involving subject matter experts, the maturity model was refined. The resulting maturity model was implemented in a case study within a national regulatory authority context, to determine the generalisability and empirical validity of the model. The model was deemed a useful, unique, and valuable contribution to organisations operating SRSs, having achieved the stated aim.AFRIKAANSE OPSOMMING: Oor die algemeen word dit aanvaar dat alle medisyne die potensiaal het om negatiewe geneesmiddelreaksies te veroorsaak tydens hul normale terapeutiese gebruik. Dwelmveiligheidswaarneming tydens die na-bemarkingsmagtigingsfase genereer die meerderheid van dwelmveiligheidsdata, selfs meer as die kliniese proewe tydens die dwelmontwikkelingsproses. Pharmacovigilance (PV), is gebaseer op die mediese evaluering van geneesmiddelreaksies of dwelmverwante probleme, wat binne georganiseerde gesondheidsprogramme ingesamel word. PV-stelsels is inherent kompleks. Die groot aantal, fragmentasie en kompleksiteit van bestaande PV-stelsels, die ewe groot aantal belanghebbendes binne hierdie stelsels (d.w.s. farmaseutiese maatskappye, staatsregulerende owerhede, nasionale en internasionale kliniese reguleringsliggame, gesondheidswerkers, ens.), sowel as die beduidende aantal dimensies waarlangs die effektiwiteit en doeltreffendheid beinvloed kan word en ook gemeet word, dra by tot hierdie kompleksiteit. Die primeere doel van enige PV-stelsel is om pasieentveiligheid te verbeter en te beskerm deur gesondheidsorgwerkers in staat te stel om meer ingeligte terapeutiese besluite te neem. Die bereiking van hierdie doelwit is afhanklik van die suksesvolle kommunikasie van relevante geneesmiddelreaksie-inligting van die pasient na die betrokke PV-owerheid. Een sodanige kommunikasiemetode is die spontane rapportering van geneesmiddelreaksies, wat algemeen beskou word as die hoeksteen van data-opwekking in PV tydens die veiligheidswaarneming na bemarking. Tans word spontane rapporteringstelsels (SRSe) gekonfronteer met probleme soos onder-rapportering en die kommunikasie van onvolledige, nie-verteenwoordigende en onbeheerde data. Die gebrek aan standardisering en interoperabiliteit tussen hierdie stelsels lei tot 'n verminderde vermoe om nuwe geneesmiddelsinteraksies en geneesmiddelreaksies na te spoor en te karakteriseer. Die primere struikelblok vir die bereiking van interoperabiliteit tussen SRSe is die fundamentele verskil in die doel van die bestaande SRSe. Belanghebbendes in die PV-stelsel bedryf SRSe met verskillende doelwitte en perspektiewe, insluitende: die handhawing van regulatoriese vereistes; die vermindering van finansiele risiko; en die beskerming en bevordering van pasientveiligheid in openbare gesondheidsorgprogramme. Die doel van hierdie studie is om by te dra tot die interoperabiliteit van SRSe in die PV-landskap deur die ontwikkeling van 'n nuwe en oorspronklike volwassenheidsmodel met 'n sosiotegniese stelselfokus. Die doel van die model is om interoperabiliteit te bevorder en te verbeter deur die mate van integrasie van betrokke stelsels aan te spreek, leiding te gee oor watter stelselkomponente verbeter moet word, asook om die interoperabiliteitsprogressies in die wereldwye gemeenskap van SRSe te meet. 'n Multidissiplinere literatuuroorsig oor PV, volwassenheidsmodelle, interoperabiliteit en sosiotegniese-stelsels het gedien as 'n teoretiese grondslag vir die ontwikkeling van die model. Die ontwikkeling van die model het gevolg op 'n aanpassing van die 8-fase prosedure model vir die ontwikkeling van volwassenheidsmodelle, voorgestel deur Becker et al. (2009). 'n Vergelyking van 18 bestaande volwassenheidsmodelle in die velde van: (i) PV; (ii) eHealth; (iii) eHealth/interoperabiliteit; (iv) interoperabiliteit; en (v) IT-infrastruktuur, is uitgevoer. Die model bestaan uit drie domeine, sewe subdomeine en dertig dimensies wat geidentifiseer is as gevolg van die voorafgaande literatuuroorsig en vergelyking van bestaande modelle. Die volwassenheidsmodel is deur 'n kombinasie van verifierings- en valideringsprosesse, waar vakkundiges betrokke was, verfyn. Om die veralgemeenbaarheid en empiriese geldigheid van die model te bepaal, is die gevolglike volwassenheidsmodel in 'n gevallestudie, binne die nasionale konteks van 'n regulerende owerheid, geimplementeer. Die gestelde doel van die model is bereik en die model word geag as 'n nuttige, unieke en waardevolle bydrae tot organisasies wat SRS'e bedryf.Doctora

Under-reporting of Adverse Drug Reactions to the Food & Drug Administration

This study examined the potential significant differences in the distribution of adverse drug reactions (ADRs) by reporter (consumer versus physician) and patient outcome at case and event level. This study also contains exploratory questions to evaluate reporting of ADRs by consumers versus physician by system organ class (SOC) and reporter demographics within the United States Food & Drug Administration Adverse Event Reporting System (FAERS). The theoretical foundation applied in this quantitative study was the social amplification of risk framework. Data from the second quarter of 2016 were obtained from FAERS, and a total of 87,807 ADR reports corresponding to 143,399 ADRs were analyzed by utilizing the chi-square test, the odds ratio, and logistic regression. Cross-sectional design was employed to compare reporting of ADRs at the case and event level (case-based and event-based analyses, respectively) between reporters (consumer versus physician), specifically, for patient outcome, as well as SOC and reporter demographics. For both the case-based and event-based analyses, findings revealed that consumers reported more serious ADRs in comparison to physicians. Furthermore, findings confirmed a difference in ADR reporting between consumers and physicians depending on SOC groups. Additionally, consumers reported more nonserious ADRs in comparison to physicians. The results from this study may have implications for positive social change by augmenting pharmacovigilance systems at a national and international level to identify risks and risk factors spontaneously reported after drugs have been on the market

In Silico Toxicology Data Resources to Support Read-Across and (Q)SAR

A plethora of databases exist online that can assist in in silico chemical or drug safety assessment. However, a systematic review and grouping of databases, based on purpose and information content, consolidated in a single source has been lacking. To resolve this issue, this review provides a comprehensive listing of the key in silico data resources relevant to: chemical identity and properties, drug action, toxicology (including nano-material toxicity), exposure, omics, pathways, Absorption, Distribution, Metabolism and Elimination (ADME) properties, clinical trials, pharmacovigilance, patents-related databases, biological (genes, enzymes, proteins, other macromolecules etc.) databases, protein-protein interactions (PPIs), environmental exposure related, and finally databases relating to animal alternatives in support of 3Rs policies. More than nine hundred databases were identified and reviewed against criteria relating to accessibility, data coverage, interoperability or application programming interface (API), appropriate identifiers, types of in vitro-in vivo -clinical data recorded and suitability for modelling, read-across or similarity searching. This review also specifically addresses the need for solutions for mapping and integration of databases into a common platform for better translatability of preclinical data to clinical data

Vaccine semantics : Automatic methods for recognizing, representing, and reasoning about vaccine-related information

Post-marketing management and decision-making about vaccines builds on the early detection of safety concerns and changes in public sentiment, the accurate access to established evidence, and the ability to promptly quantify effects and verify hypotheses about the vaccine benefits and risks. A variety of resources provide relevant information but they use different representations, which makes rapid evidence generation and extraction challenging. This thesis presents automatic methods for interpreting heterogeneously represented vaccine information. Part I evaluates social media messages for monitoring vaccine adverse events and public sentiment in social media messages, using automatic methods for information recognition. Parts II and III develop and evaluate automatic methods and res