FRAME: Framework for Real‐World Evidence Assessment to Mitigate Evidence uncertainties for efficacy/effectiveness – an evaluation of regulatory and health technology assessment decision making

Abstract

Real‐World Evidence (RWE) is increasingly used in submissions to regulatory agencies and health technology assessment bodies (HTAbs) to support the efficacy and effectiveness of new medicines and indications. However, there is limited information on the RWE characteristics that impact its role in approval and reimbursement decisions. To investigate these characteristics, we developed FRAME: a Framework for Real‐world evidence Assessment to Mitigate Evidence uncertainties for efficacy/effectiveness. We compiled a list of medicinal product indications where RWE supported the efficacy of interventional trials or assessed effectiveness in observational settings. FRAME was applied to a prioritized subset of these submissions to authorities from North America, Europe, and Australia. For each product indication, we extracted information on characteristics describing the submission, clinical context, strength of evidence, and process factors from publicly available assessment reports. Of the 87 identified medicinal product indications, 15 were prioritized, covering 68 submissions and 76 RWE studies across 11 authorities in scope. Four main results emerged: (i) low granularity within assessment reports on the analyzed variables, limiting the learnings from analyzing them; (ii) variability in how RWE was assessed within and across regulatory agencies and HTAbs; (iii) a positive association between the proportion of positive comments from authorities on RWE studies and their impact on decision making. Particularly, a large effect size was consistently noted when RWE was considered primary evidence; and (iv) limited use of advanced RWE study designs. These findings support five recommendations to enhance shared learning on RWE, clarify its evidentiary value, and generate evidence to better support authorities' decision making