Comparing ROC Curves Derived From Regression Models

In constructing predictive models, investigators frequently assess the incremental value of a predictive marker by comparing the ROC curve generated from the predictive model including the new marker with the ROC curve from the model excluding the new marker. Many commentators have noticed empirically that a test of the two ROC areas often produces a non-significant result when a corresponding Wald test from the underlying regression model is significant. A recent article showed using simulations that the widely-used ROC area test [1] produces exceptionally conservative test size and extremely low power [2]. In this article we show why the ROC area test is invalid in this context. We demonstrate how a valid test of the ROC areas can be constructed that has comparable statistical properties to the Wald test. We conclude that using the Wald test to assess the incremental contribution of a marker remains the best strategy. We also examine the use of derived markers from non-nested models and the use of validation samples. We show that comparing ROC areas is invalid in these contexts as well

Inferential Methods to Assess the Difference in the Area Under the Curve From Nested Binary Regression Models

The area under the curve (AUC) is the most common statistical approach to evaluate the discriminatory power of a set of factors in a binary regression model. A nested model framework is used to ascertain whether the AUC increases when new factors enter the model. Two statistical tests are proposed for the difference in the AUC parameters from these nested models. The asymptotic null distributions for the two test statistics are derived from the scenarios: (A) the difference in the AUC parameters is zero and the new factors are not associated with the binary outcome, (B) the difference in the AUC parameters is less than a strictly positive value. A confidence interval for the difference in AUC parameters is developed. Simulations are generated to determine the finite sample operating characteristics of the tests and a pancreatic cancer data example is used to illustrate this approach

A Metastasis or a Second Independent Cancer? Evaluating the Clonal Origin of Tumors Using Array-CGH Data

When a cancer patient develops a new tumor it is necessary to determine if this is a recurrence (metastasis) of the original cancer, or an entirely new occurrence of the disease. This is accomplished by assessing the histo-pathology of the lesions, and it is frequently relatively straightforward. However, there are many clinical scenarios in which this pathological diagnosis is difficult. Since each tumor is characterized by a genetic fingerprint of somatic mutations, a more definitive diagnosis is possible in principle in these difficult clinical scenarios by comparing the fingerprints. In this article we develop and evaluate a statistical strategy for this comparison when the data are derived from array comparative genomic hybridization, a technique designed to identify all of the somatic allelic gains and losses across the genome. Our method involves several stages. First a segmentation algorithm is used to estimate the regions of allelic gain and loss. Then the broad correlation in these patterns between the two tumors is assessed, leading to an initial likelihood ratio for the two diagnoses. This is then further refined by comparing in detail each plausibly clonal mutation within individual chromosome arms, and the results are aggregated to determine a final likelihood ratio. The method is employed to diagnose patients from several clinical scenarios, and the results show that in many cases a strong clonal signal emerges, occasionally contradicting the clinical diagnosis. The “quality” of the arrays can be summarized by a parameter that characterizes the clarity with which allelic changes are detected. Sensitivity analyses show that most of the diagnoses are robust when the data are of high quality

Extrapleural pneumonectomy versus pleurectomy/decortication in the surgical management of malignant pleural mesothelioma: Results in 663 patients

ObjectiveThe optimal procedure for resection of malignant pleural mesothelioma is controversial, partly because previous analyses include small numbers of patients. We performed a multi-institutional study to increase statistical power to detect significant differences in outcome between extrapleural pneumonectomy and pleurectomy/decortication.MethodsPatients with malignant pleural mesothelioma who underwent extrapleural pneumonectomy or pleurectomy/decortication at 3 institutions were identified. Survival and prognostic factors were analyzed by the Kaplan–Meier method, log-rank test, and Cox proportional hazards analysis.ResultsFrom 1990 to 2006, 663 consecutive patients (538 men and 125 women) underwent resection. The median age was 63 years (range, 26–93 years). The operative mortality was 7% for extrapleural pneumonectomy (n = 27/385) and 4% for pleurectomy/decortication (n = 13/278). Significant survival differences were seen for American Joint Committee on Cancer stages 1 to 4 (P < .001), epithelioid versus non-epithelioid histology (P < .001), extrapleural pneumonectomy versus pleurectomy/decortication (P < .001), multimodality therapy versus surgery alone (P < .001), and gender (P < .001). Multivariate analysis demonstrated a hazard rate of 1.4 for extrapleural pneumonectomy (P < .001) controlling for stage, histology, gender, and multimodality therapy.ConclusionPatients who underwent pleurectomy/decortication had a better survival than those who underwent extrapleural pneumonectomy; however, the reasons are multifactorial and subject to selection bias. At present, the choice of resection should be tailored to the extent of disease, patient comorbidities, and type of multimodality therapy planned

Mitochondrial DNA copy number variation across human cancers.

Mutations, deletions, and changes in copy number of mitochondrial DNA (mtDNA), are observed throughout cancers. Here, we survey mtDNA copy number variation across 22 tumor types profiled by The Cancer Genome Atlas project. We observe a tendency for some cancers, especially of the bladder, breast, and kidney, to be depleted of mtDNA, relative to matched normal tissue. Analysis of genetic context reveals an association between incidence of several somatic alterations, including IDH1 mutations in gliomas, and mtDNA content. In some but not all cancer types, mtDNA content is correlated with the expression of respiratory genes, and anti-correlated to the expression of immune response and cell-cycle genes. In tandem with immunohistochemical evidence, we find that some tumors may compensate for mtDNA depletion to sustain levels of respiratory proteins. Our results highlight the extent of mtDNA copy number variation in tumors and point to related therapeutic opportunities

Genomic investigation of etiologic heterogeneity: methodologic challenges

Background: The etiologic heterogeneity of cancer has traditionally been investigated by comparing risk factor frequencies within candidate sub-types, defined for example by histology or by distinct tumor markers of interest. Increasingly tumors are being profiled for molecular features much more extensively. This greatly expands the opportunities for defining distinct sub-types. In this article we describe an exploratory analysis of the etiologic heterogeneity of clear cell kidney cancer. Data are available on the primary known risk factors for kidney cancer, while the tumors are characterized on a genome-wide basis using expression, methylation, copy number and mutational profiles. Methods: We use a novel clustering strategy to identify sub-types. This is accomplished independently for the expression, methylation and copy number profiles. The goals are to identify tumor sub-types that are etiologically distinct, to identify the risk factors that define specific sub-types, and to endeavor to characterize the key genes that appear to represent the principal features of the distinct sub-types. Results: The analysis reveals strong evidence that gender represents an important factor that distinguishes disease sub-types. The sub-types defined using expression data and methylation data demonstrate considerable congruence and are also clearly correlated with mutations in important cancer genes. These sub-types are also strongly correlated with survival. The complexity of the data presents many analytical challenges including, prominently, the risk of false discovery. Conclusions: Genomic profiling of tumors offers the opportunity to identify etiologically distinct sub-types, paving the way for a more refined understanding of cancer etiology. Electronic supplementary material The online version of this article (doi:10.1186/1471-2288-14-138) contains supplementary material, which is available to authorized users

Positron-emission tomography–based staging reduces the prognostic impact of early disease progression in patients with follicular lymphoma

Background: Previous studies reported that early progression of disease (POD) after initial therapy predicted poor overall survival (OS) in patients with follicular lymphoma (FL). Here, we investigated whether pre-treatment imaging modality had an impact on prognostic significance of POD. Methods: In this retrospective study, we identified 1088 patients with grade I–IIIA FL; of whom, 238 patients with stage II–IV disease were initially treated with rituximab, cyclophosphamide, doxorubicin, vincristine and prednisone (R-CHOP), and 346 patients were treated with rituximab-based chemotherapy. Patients (N = 484) from the FOLL05 study served as an independent validation cohort. We risk-stratified patients based on pre-treatment radiographic imaging (positron-emission tomography [PET] versus computed tomography [CT]) and early POD status using event-defining and landmark analyses. A competing risk analysis evaluated the association between early POD and histologic transformation. Results: In the discovery cohort, patients with POD within 24 months (PFS24) of initiating R-CHOP therapy had a 5-year OS of 57.6% for CT-staged patients compared with 70.6% for PET-staged patients. In the validation cohort, the 5-year OS for patients with early POD was 53.9% and 100% in CT- and PET-staged patients, respectively. The risk of histologic transformation in patients whose disease progressed within one year of initiating therapy was higher in CT-staged patients than in PET-staged patients (16.7% versus 6.3%, respectively), which was associated with a 9.7-fold higher risk of death. Conclusion: In FL, pre-treatment PET staging reduced the prognostic impact of early POD compared with CT staging. Patients with early POD and no histologic transformation have an extended OS with standard therapy

InterMEL: An international biorepository and clinical database to uncover predictors of survival in early-stage melanoma

We are conducting a multicenter study to identify classifiers predictive of disease-specific survival in patients with primary melanomas. Here we delineate the unique aspects, challenges, and best practices for optimizing a study of generally small-sized pigmented tumor samples including primary melanomas of at least 1.05mm from AJTCC TNM stage IIA-IIID patients. We also evaluated tissue-derived predictors of extracted nucleic acids’ quality and success in downstream testing. This ongoing study will target 1,000 melanomas within the international InterMEL consortium.Medicin