Statistical issues in trials of preexposure prophylaxis

Purpose of review We discuss selected statistical issues in the design and analysis of preexposure prophylaxis (PrEP) trials. The general principles may inform thinking for other interventions in HIV prevention. Recent findings To date, four different designs have been used to determine the effectiveness of PrEP: randomized, doubleblind, placebo-controlled; randomized, open-label, immediate or delayed access; nonrandomized comparison of HIV incidence according to the level of drug detected; comparison of the observed HIV incidence to the expected rate using historical control data. Open-label trials of PrEP, which assess public health effectiveness, complement the placebo-controlled trials which established the biological efficacy of TDF/ FTC. Future trials of PrEP will be highly challenging to design since a no PrEP group is difficult to justify and the natural control regimen, TDF/FTC, is highly efficacious. Summary Standard statistical paradigms for noninferiority trials should be reconsidered for evaluating alternative PrEP regimens

Confidence limits for the averted infections ratio estimated via the counterfactual placebo incidence rate

Objectives The averted infections ratio (AIR) is a novel measure for quantifying the preservation-of-effect in active-control non-inferiority clinical trials with a time-to-event outcome. In the main formulation, the AIR requires an estimate of the counterfactual placebo incidence rate. We describe two approaches for calculating confidence limits for the AIR given a point estimate of this parameter, a closed-form solution based on a Taylor series expansion (delta method) and an iterative method based on the profile-likelihood. Methods For each approach, exact coverage probabilities for the lower and upper confidence limits were computed over a grid of values of (1) the true value of the AIR (2) the expected number of counterfactual events (3) the effectiveness of the active-control treatment. Results Focussing on the lower confidence limit, which determines whether non-inferiority can be declared, the coverage achieved by the delta method is either less than or greater than the nominal coverage, depending on the true value of the AIR. In contrast, the coverage achieved by the profile-likelihood method is consistently accurate. Conclusions The profile-likelihood method is preferred because of better coverage properties, but the simpler delta method is valid when the experimental treatment is no less effective than the control treatment. A complementary Bayesian approach, which can be applied when the counterfactual incidence rate can be represented as a prior distribution, is also outlined

A Bayesian averted infection framework for PrEP trials with low numbers of HIV infections: application to the results of the DISCOVER trial

Trials of candidate agents for HIV pre-exposure prophylaxis (PrEP) might randomly assign participants to be given a new PrEP agent or oral coformulated tenofovir disoproxil fumarate plus emtricitabine. This design presents unique challenges in interpretation. First, with two active arms, HIV incidence might be low. Second, the effectiveness of tenofovir disoproxil fumarate plus emtricitabine varies across populations; thus, similar HIV incidence between groups could be consistent with a wide range of effectiveness for the new PrEP. We propose a two-part approach to trial results. First, we use Bayesian methods to incorporate assumptions about the background incidence of HIV in the trial in the absence of PrEP, possibly augmented by external data. On the basis of the estimated background incidence, we estimate and compare the number of averted (or prevented) HIV infections in each of the two trial groups, calculating the averted infections ratio. We apply these methods to a completed trial of tenofovir alafenamide plus emtricitabine for PrEP. Our framework shows that leveraging external information to estimate averted infections and the averted infections ratio enhances the efficiency and interpretation of active-controlled PrEP trials

Mosaic effectiveness: measuring the impact of novel PrEP methods

Various ongoing trials seek to evaluate long-acting pre-exposure prophylaxis (PrEP) agents by showing that they are non-inferior to daily oral tenofovir disoproxil fumarate and emtricitabine. Trials comparing oral PrEP to new methods examine effectiveness in a setting where only one or the other is provided; however, a new product will probably be delivered in a context where oral PrEP is also available. The effectiveness of a new PrEP product is best measured by its potential effect in a context that also includes oral tenofovir disoproxil fumarate and emtricitabine as an option. We offer an alternative standard for long-acting products—a measure of the effectiveness of the new product in addition to oral tenofovir disoproxil fumarate and emtricitabine as compared with oral PrEP alone. We term this measure mosaic effectiveness. We illustrate scenarios where a novel product can fail to show non-inferiority but show substantial mosaic effectiveness, thus implying the public health value of the novel product even if it is less effective than oral PrEP. Regulatory standards should consider mosaic effectiveness, not just comparative effectiveness. We assert that measurements that combine rigor with public health relevance can accelerate progress against the HIV epidemic

Incidence of symptomatic toxoplasma eye disease: aetiology and public health implications.

Ocular disease is the commonest disabling consequence of toxoplasma infection. Incidence and lifetime risk of ocular symptoms were determined by ascertaining affected patients in a population-based, active reporting study involving ophthalmologists serving a population of 7.4 million. Eighty-seven symptomatic episodes were attributed to toxoplasma infection. Bilateral visual acuity of 6/12 or less was found in seven episodes (8%) and was likely to have been transient in most cases. Black people born in West Africa had a 100-fold higher incidence of symptoms than white people born in Britain. Only two patients reported symptoms before 10 years of age. The estimated lifetime risk of symptoms in British born individuals (52% of all episodes) was 18/100000 (95% confidence interval: 10.8-25.2). The low risk and mild symptoms in an unscreened British population indicate limited potential benefits of prenatal or postnatal screening. The late age at presentation suggests a mixed aetiology of postnatally acquired and congenital infection for which primary prevention may be appropriate, particularly among West Africans

The Connection between the Averted Infections Ratio and the Rate Ratio in Active-control Trials of Pre-exposure Prophylaxis Agents

The design and analysis of active-control trials to evaluate experimental HIV pre-exposure prophylaxis (PrEP) agents pose serious statistical challenges. We recently proposed a new outcome measure, the averted infections ratio (AIR) – the proportion of infections that would be averted by using the experimental agent rather than the control agent (compared to no intervention). The main aim of the current paper is to examine the mathematical connection between AIR and the HIV incidence rate ratio, the standard outcome measure. We also consider the sample size implications of the choice of primary outcome measure and explore the connection between effectiveness and efficacy under a simplified model of adherence

Superiority and non-inferiority: two sides of the same coin?

BACKGROUND: The classification of phase 3 trials as superiority or non-inferiority has become routine, and it is widely accepted that there are important differences between the two types of trial in their design, analysis and interpretation. MAIN TEXT: There is a clear rationale for the superiority/non-inferiority framework in the context of regulatory trials. The focus of our article is non-regulatory trials with a public health objective. First, using two examples from infectious disease research, we show that the classification of superiority or non-inferiority trials is not always straightforward. Second, we show that several arguments for different approaches to the design, analysis and interpretation of superiority and non-inferiority trials are unconvincing when examined in detail. We consider, in particular, the calculation of sample size (and the choice of delta or the non-inferiority margin), intention-to-treat versus per-protocol analyses, and one-sided versus two-sided confidence intervals. We argue that the superiority/non-inferiority framework is not just unnecessary but can have a detrimental effect, being a barrier to clear scientific thought and communication. In particular, it places undue emphasis on tests for significance or non-inferiority at the expense of estimation. We emphasise that these concerns apply to phase 3 non-regulatory trials in general, not just to those where the classification of the trial as superiority or non-inferiority is ambiguous. CONCLUSIONS: Guidelines and statistical practice should abandon the sharp division between superiority and non-inferiority phase 3 non-regulatory trials and be more closely aligned to the clinical and public health questions that motivate the trial

The averted infections ratio: a novel measure of effectiveness of experimental HIV pre-exposure prophylaxis agents

Tenofovir disoproxil fumarate combined with emtricitabine is a highly effective oral pre-exposure prophylaxis (PrEP) agent for preventing the acquisition of HIV. This effectiveness has consequences for the design and analysis of trials assessing experimental PrEP regimens, which now generally include an active-control tenofovir disoproxil fumarate plus emtricitabine group, rather than a placebo group, as a comparator. Herein, we describe major problems in the interpretation of the primary measure of effectiveness proposed for these trials, namely the ratio of HIV incidence in the experimental agent group to that in the active-control group. We argue that valid interpretation requires an assumption about one of two parameters: either the incidence among trial participants had they not received PrEP or the effectiveness of tenofovir disoproxil fumarate plus emtricitabine within the trial. However, neither parameter is directly observed because of the absence of a no-treatment group, thus requiring the use of external evidence or subjective judgment. We propose an alternative measure of effectiveness based on the concept of averted infections, which incorporates one of these parameters. The measure is simple to interpret, has clinical and public health relevance, and is a natural preservation-of-effect criterion for assessing statistical non-inferiority. Its adoption could also allow the use of smaller sample sizes, currently a major barrier to the assessment of experimental PrEP regimen

Cost Effectiveness of Protease Inhibitor Monotherapy Versus Standard Triple Therapy in the Long-Term Management of HIV Patients: Analysis Using Evidence from the PIVOT Trial

BACKGROUND: Protease inhibitor (PI) monotherapy can maintain virological suppression in the majority of patients once it has been established on triple therapy and may also have the potential for substantial cost savings arising from the use of fewer drugs. However, the cost effectiveness of PI monotherapy has yet to be demonstrated. OBJECTIVES: In this study we examine the cost effectiveness of PI monotherapy with prompt return to combination therapy in the event of viral load rebound compared with ongoing triple therapy (OT) in patients with suppressed viral load on combination antiretroviral therapy (ART) in the UK. METHODS: The analysis used data from the PIVOT trial in which HIV-positive adults with suppressed viral load for ≥24 weeks on combination ART were randomised to maintain OT or to a strategy of PI monotherapy with prompt return to combination therapy if viral load rebounded. A cost-effectiveness analysis including long-term modelling was conducted. Main outcomes included UK National Health Service (NHS) costs and quality-adjusted life-years (QALYs) with comparative results presented as incremental cost-effectiveness ratios. RESULTS: PI monotherapy was cost saving as a result of large savings in ART drug costs while being no less effective in terms of QALYs in the within-trial analysis and marginally less effective with lifetime modelling. In the base-case analysis over 3 years, the incremental total cost per patient was -£6424.11 (95 % confidence interval -7418.84 to -5429.38) and incremental QALYs were 0.0051 (95 % CI -0.0479 to 0.0582), resulting in PI monotherapy 'dominating' OT. Multiple scenario analyses found that PI monotherapy was cost saving with no marked differences in QALYs. Modelling of lifetime costs and QALYs showed that PI monotherapy was associated with significant cost savings and was marginally less effective; PI monotherapy was cost effective at accepted cost-effectiveness thresholds in all but one scenario analysis. CONCLUSIONS: Under most assumptions, PI monotherapy appears to be a cost-effective treatment strategy compared with OT for HIV-infected patients who have achieved sustained virological suppression

HIV prevention trial design in an era of effective pre-exposure prophylaxis

Pre-exposure prophylaxis (PrEP) has demonstrated remarkable effectiveness protecting at-risk individuals from HIV-1 infection. Despite this record of effectiveness, concerns persist about the diminished protective effect observed in women compared with men and the influence of adherence and risk behaviors on effectiveness in targeted subpopulations. Furthermore, the high prophylactic efficacy of the first PrEP agent, tenofovir disoproxil fumarate/emtricitabine (TDF/FTC), presents challenges for demonstrating the efficacy of new candidates. Trials of new agents would typically require use of non-inferiority (NI) designs in which acceptable efficacy for an experimental agent is determined using pre-defined margins based on the efficacy of the proven active comparator (i.e. TDF/FTC) in placebo-controlled trials. Setting NI margins is a critical step in designing registrational studies. Under- or over-estimation of the margin can call into question the utility of the study in the registration package. The dependence on previous placebo-controlled trials introduces the same issues as external/historical controls. These issues will need to be addressed using trial design features such as re-estimated NI margins, enrichment strategies, run-in periods, crossover between study arms, and adaptive re-estimation of sample sizes. These measures and other innovations can help to ensure that new PrEP agents are made available to the public using stringent standards of evidence