Sequential double cross-validation for assessment of added predictive ability in high-dimensional omic applications

Enriching existing predictive models with new biomolecular markers is an important task in the new multi-omic era. Clinical studies increasingly include new sets of omic measurements which may prove their added value in terms of predictive performance. We introduce a two-step approach for the assessment of the added predictive ability of omic predictors, based on sequential double cross-validation and regularized regression models. We propose several performance indices to summarize the two-stage prediction procedure and a permutation test to formally assess the added predictive value of a second omic set of predictors over a primary omic source. The performance of the test is investigated through simulations. We illustrate the new method through the systematic assessment and comparison of the performance of transcriptomics and metabolomics sources in the prediction of body mass index (BMI) using longitudinal data from the Dietary, Lifestyle, and Genetic determinants of Obesity and Metabolic syndrome (DILGOM) study, a population-based cohort from Finland

Assessment of the consistency and robustness of results from a multicenter trial of remission maintenance therapy for acute myeloid leukemia

<p>Abstract</p> <p>Background</p> <p>Data from a randomized multinational phase 3 trial of 320 adults with acute myeloid leukemia (AML) demonstrated that maintenance therapy with 3-week cycles of histamine dihydrochloride plus low-dose interleukin-2 (HDC/IL-2) for up to 18 months significantly improved leukemia-free survival (LFS) but lacked power to detect an overall survival (OS) difference.</p> <p>Purpose</p> <p>To assess the consistency of treatment benefit across patient subsets and the robustness of data with respect to trial centers and endpoints.</p> <p>Methods</p> <p>Forest plots were constructed with hazard ratios (HRs) of HDC/IL-2 treatment effects versus no treatment (control) for prospectively defined patient subsets. Inconsistency coefficients (I²) and interaction tests (X²) were used to detect any differences in benefit among subsets. Robustness of results to the elimination of individual study centers was performed using "leave-one-center-out" analyses. Associations between treatment effects on the endpoints were evaluated using weighted linear regression between HRs for LFS and OS estimated within countries.</p> <p>Results</p> <p>The benefit of HDC/IL-2 over controls was statistically consistent across all subsets defined by baseline prognostic variables. I²and <it>P</it>-values of X²ranged from 0.00 to 0.51 and 0.14 to 0.91, respectively. Treatment effects were statistically significant in 14 of 28 subsets analyzed. The "leave-one-center-out" analysis confirmed that no single center dominated (<it>P</it>-values ranged from 0.004 to 0.020 [mean 0.009]). The HRs representing the HDC/IL-2 effects on LFS and OS were strongly correlated at the country level (R²= 0.84).</p> <p>Limitations</p> <p>Small sample sizes in some of the subsets analyzed.</p> <p>Conclusions</p> <p>These analyses confirm the consistency and robustness of the HDC/IL-2 effect as compared with no treatment. LFS may be an acceptable surrogate for OS in future AML trials. Analyses of consistency and robustness may aid interpretation of data from multicenter trials, especially in populations with rare diseases, when the size of randomized clinical trials is limited.</p> <p>Trial Registration</p> <p>ClinicalTrials.gov: <a href="http://www.clinicaltrials.gov/ct2/show/NCT00003991">NCT00003991</a></p

Use of surrogate endpoints in healthcare policy : proposal for consistent adoption of a validation framework

We propose a three step framework for the evaluation of surrogate endpoints for health policy decisions on health technologies

Relationship between effects on time-to-disease progression and overall survival in studies of metastatic breast cancer

The relationship between overall survival (OS) and disease progression end points has been demonstrated in colorectal, colon, and non-small cell lung cancers. We assessed the association between OS and time-to-progression (TTP) or progression-free survival (PFS) in metastatic breast cancer (MBC) studies. A literature search retrieved all randomised controlled trials since 1994 in patients with MBC in which OS and either TTP or PFS were reported. Summary data on trial and patient characteristics were abstracted. Study effect sizes were derived as the ratio of median progression (or survival) times, which approximates the hazard ratio. Effects were centred at zero for regression analyses weighted by sample size. Numerous treatments were represented in 67 studies (17 081 patients). Modeling showed a positive association between outcomes for progression and survival (R2=0.30) with a slope of 0.32 (P<0.001) and a non-significant intercept. Thus, a treatment effect on TTP/PFS translated into a concordant effect on OS, but with attenuated effect size. Similar results were found in models of subsets and sensitivity analyses. These results demonstrate that treatment effects on progression end points in MBC trials are expected to result in treatment differences on OS that are smaller yet consistently in the same direction

Therapeutic aims of drugs offering only progression-free survival are misunderstood by patients, and oncologists may be overly optimistic about likely benefits

PURPOSE: The use of novel and often expensive drugs offering limited survival benefit in advanced disease is controversial. Treatment recommendations are influenced by patient characteristics and trial data showing overall response rates (ORR), progression-free survival (PFS) and overall survival (OS). PFS is frequently the primary outcome in licencing studies. PATIENTS AND METHODS: As part of a longitudinal study Assessing the 'VALue' to patients of PROgression Free Survival (AVALPROFS), oncologists completed checklists at baseline following consultations with patients. Questions probed perceived clinical benefits of the drugs to populations in general. Patients completed study-specific interview schedules at baseline, 6 weeks into treatment, and at withdrawal due to toxicity or progression. Patients also completed tumour- and treatment-specific quality of life questionnaires monthly for their time in the study. Only baseline results are reported here. RESULTS: Thirty-two UK oncologists discussed management options with 90 patients with heterogeneous advanced cancers. Oncologists' estimates of medical benefit in general from treatment varied between 10 and 80 %. They expected 46/90 (51 %) of their patients to derive some clinical benefit from the prescribed treatment but were either unsure or expected none for 44/90 (49 %). Predictions of life expectancy were variable but 62 % (56/90) of patients were expected to survive longer with treatment. A majority of patients 51/90 (57 %) had 'no idea' or were 'unclear' what PFS meant and 45/90 (50 %) thought extension of life was the primary therapeutic aim of treatment. CONCLUSION: Discussions between doctors and patients with metastatic disease about future management plans and likely therapeutic gains are challenging. Factors influencing decisions about putative benefits of novel drugs are often applied inconsistently can be overly optimistic and may even contradict published data