Prioritizing research: the use of risk prediction, value of information analyses, and portfolio evaluation to improve public investments in cancer clinical trials
Thesis (Ph.D.)--University of Washington, 2014Background: Fewer than half of all medical interventions in use today are supported by clinical evidence. Despite allocating more than $11 billion each year to support clinical research, federal funding for medical research lacks a coordinated system for prioritizing and allocating resources to efficiently address these important knowledge gaps. The Institute of Medicine recently stressed that rigorous prioritization of trial concepts for large cancer clinical trials cooperative groups was critical to ensure that limited public funds are used effectively and efficiently. Yet despite this ardent call to improve the means of prioritizing and selecting cancer clinical trials, many questions remain about how to achieve these goals. Portfolio management is a systematic approach to decision-making that is widely used in the private sector to inform and manage research investments. Yet despite it's conceptual simplicity, this approach has not been used to inform publicly funded cancer clinical trial investments because of the difficulty in defining and measuring risk and return in this setting. In this dissertation, I therefore developed and evaluated quantitative measures of risk and return that were appropriate for National Clinical Trials Network-sponsored trials and applied a proof of concept portfolio evaluation approach to a sample of clinical trial proposals recently reviewed by a large cancer clinical trials cooperative group, SWOG. Methods: In Chapter I, I developed a statistical model to predict the risk of an accrual feasibility failure, defined as a trial that does not enroll a sufficient number of patients and consequently is unable to inform clinical practice patterns, based on trial-level variables available before the trial is launched, and evaluated its internal validity. In Chapter II, in collaboration with key stakeholders I developed a process to efficiently quantify the societal return of the proposed studies using Value of Information (VOI) methods and evaluated its feasibility and acceptability. Lastly, in Chapter III, I estimated the predicted risk and expected return for a sample of recently reviewed clinical trial proposals to illustrate how a portfolio management framework could inform funding decisions within a cancer clinical trials cooperative group setting. Results: In Chapter I, I provide a comprehensive and empirical assessment of risk factors that are associated with and predictive of a clinical trial that does not meet 50% of its target accrual. I identified several novel predictors, and showed that these predictors in combination with several established risk factors could predict which NCTN-sponsored clinical trials were at highest risk of poor accrual. In Chapter II, I describe several key changes that I made to the traditional Value of Information analysis framework to accommodate SWOG stakeholders preferences and facilitate timely calculation. The efficient and pragmatic process that I developed leveraged information included in each trial proposal and reported the expected health benefits and incremental healthcare costs associated with acquiring additional information separately. In Chapter III, I illustrate how a portfolio management approach provides a means of efficiently summarizing both the expected accrual feasibility and societal return - two critical criteria - for a large sample of trial proposals simultaneously and therefore provides a framework for evaluating trial concepts against one another. Conclusions: I found that a portfolio evaluation is a feasible and potentially useful response to the IOM's call for more systematic approaches to select and prioritize trial concepts against one another. The approach can facilitate the ranking of a large number of trial concepts simultaneously using two key criteria for which I developed novel methods to estimate, and also inform longer-term strategic decision-making. A portfolio evaluation approach could therefore help decision makers select and prioritize cancer clinical trial concepts that have the greatest potential to improve population health and thereby optimize the return on limited research funds
