Assessing the Accuracy of a New Diagnostic Test When a Gold Standard Does Not Exist

Often the accuracy of a new diagnostic test must be assessed when a perfect gold standard does not exist. Use of an imperfect test biases the accuracy estimates of the new test. This paper reviews existing approaches to this problem including discrepant resolution and latent class analysis. Deficiencies with these approaches are identified. A new approach is proposed that combines the results of several imperfect reference tests to define a better reference standard. We call this the composite reference standard (CRS). Using the CRS, accuracy can be assessed using multistage sampling designs. Maximum likelihood estimates of accuracy and expressions for the variance of sensitivity and specificity are provided. Data from clinical literature on the detection of Chlamydia trachomatis are used to illustrate and compare the different approaches. Advantages of the CRS relative to other approaches include that the CRS reference is explicitly defined, does not depend on the results of the new test under investigation, and is easy to interpret

Estimating Disease Prevalence in Two-Phase Studies

Disease prevalence is ideally estimated using a “gold standard” to ascertain true disease status on all subjects in a population of interest. In practice, however, the gold standard may be too costly or invasive to be applied to all subjects, in which case a two-phase design is often employed. Phase 1 data consisting of inexpensive and non-invasive screening tests on all study subjects are used to determine the subjects that receive the gold standard in the second phase. Naïve estimates of prevalence in two-phase studies can be biased (verification bias). Imputation and re-weighting estimators are often used to avoid this bias. We contrast the forms and attributes of the various prevalence estimators. Distribution theory and simulation studies are used to investigate their bias and efficiency. We conclude that the semiparametric efficient approach is the preferred method for prevalence estimation in two-phase studies. It is more robust and comparable in its efficiency to imputation and other re-weighting estimators. It is also easy to implement. We use this approach to examine the prevalence of depression in adolescents with data from the Great Smoky Mountain Study

DNA methylation-based biomarkers for early detection of non-small cell lung cancer: an update

Lung cancer is the number one cancer killer in the United States. This disease is clinically divided into two sub-types, small cell lung cancer, (10–15% of lung cancer cases), and non-small cell lung cancer (NSCLC; 85–90% of cases). Early detection of NSCLC, which is the more common and less aggressive of the two sub-types, has the highest potential for saving lives. As yet, no routine screening method that enables early detection exists, and this is a key factor in the high mortality rate of this disease. Imaging and cytology-based screening strategies have been employed for early detection, and while some are sensitive, none have been demonstrated to reduce lung cancer mortality. However, mortality might be reduced by developing specific molecular markers that can complement imaging techniques. DNA methylation has emerged as a highly promising biomarker and is being actively studied in multiple cancers. The analysis of DNA methylation-based biomarkers is rapidly advancing, and a large number of potential biomarkers have been identified. Here we present a detailed review of the literature, focusing on DNA methylation-based markers developed using primary NSCLC tissue. Viable markers for clinical diagnosis must be detectable in 'remote media' such as blood, sputum, bronchoalveolar lavage, or even exhaled breath condensate. We discuss progress on their detection in such media and the sensitivity and specificity of the molecular marker panels identified to date. Lastly, we look to future advancements that will be made possible with the interrogation of the epigenome

Bias in trials comparing paired continuous tests can cause researchers to choose the wrong screening modality

<p>Abstract</p> <p>Background</p> <p>To compare the diagnostic accuracy of two continuous screening tests, a common approach is to test the difference between the areas under the receiver operating characteristic (ROC) curves. After study participants are screened with both screening tests, the disease status is determined as accurately as possible, either by an invasive, sensitive and specific secondary test, or by a less invasive, but less sensitive approach. For most participants, disease status is approximated through the less sensitive approach. The invasive test must be limited to the fraction of the participants whose results on either or both screening tests exceed a threshold of suspicion, or who develop signs and symptoms of the disease after the initial screening tests.</p> <p>The limitations of this study design lead to a bias in the ROC curves we call <it>paired screening trial bias</it>. This bias reflects the synergistic effects of inappropriate reference standard bias, differential verification bias, and partial verification bias. The absence of a gold reference standard leads to inappropriate reference standard bias. When different reference standards are used to ascertain disease status, it creates differential verification bias. When only suspicious screening test scores trigger a sensitive and specific secondary test, the result is a form of partial verification bias.</p> <p>Methods</p> <p>For paired screening tests with bivariate normally distributed scores, we give formulae and programs to quantify the effect of <it>paired screening trial bias </it>on a paired comparison of area under the curves. We fix the prevalence of disease, and the chance a diseased subject manifests signs and symptoms. We derive the formulas for true sensitivity and specificity, and those for the sensitivity and specificity observed by the study investigator.</p> <p>Results</p> <p>The observed area under the ROC curves is quite different from the true area under the ROC curves. The typical direction of the bias is a strong inflation in sensitivity, paired with a concomitant slight deflation of specificity.</p> <p>Conclusion</p> <p>In paired trials of screening tests, when area under the ROC curve is used as the metric, bias may lead researchers to make the wrong decision as to which screening test is better.</p

Bortezomib is significantly beneficial for de novo pediatric AML patients with low phosphorylation of the NF-κB subunit RelA

Purpose: The addition of the proteasome inhibitor (PI) bortezomib to standard chemotherapy (ADE: cytarabine [Ara-C], daunorubicin, and etoposide) did not improve overall outcome of pediatric AML patients in the Children's Oncology Group AAML1031 phase 3 randomized clinical trial (AAML1031). Bortezomib prevents protein degradation, including RelA via the intracellular NF-kB pathway. In this study, we hypothesized that subgroups of pediatric AML patients benefitting from standard therapy plus bortezomib (ADEB) could be identified based on pre-treatment RelA expression and phosphorylation status. Experimental design: RelA-total and phosphorylation at serine 536 (RelA-pSer536) were measured in 483 patient samples using reverse phase protein array technology. Results: In ADEB-treated patients, low-RelA-pSer536 was favorably prognostic when compared to high-RelA-pSer536 (3-yr overall survival (OS): 81% vs. 68%, p = 0.032; relapse risk (RR): 30% vs. 49%, p = 0.004). Among low-RelA-pSer536 patients, RR significantly decreased with ADEB compared to ADE (RR: 30% vs. 44%, p = 0.035). Correlation between RelA-pSer536 and 295 other assayed proteins identified a strong correlation with HSF1-pSer326, another protein previously identified as modifying ADEB response. The combination of low-RelA-pSer536 and low-HSF1-pSer326 was a significant predictor of ADEB response (3-yr OS: 86% vs. 67%, p = 0.013). Conclusion and clinical relevance: Bortezomib may improve clinical outcome in a subgroup of AML patients identified by low-RelA-pSer536 and low-HSF1-pSer326

Heat Shock Factor 1 (HSF1-pSer326) Predicts Response to Bortezomib-Containing Chemotherapy in Pediatric AML:A COG Study

Bortezomib (BTZ) was recently evaluated in a randomized Phase 3 clinical trial which compared standard chemotherapy (cytarabine, daunorubicin, etoposide; ADE) to standard therapy with BTZ (ADEB) for de novo pediatric acute myeloid leukemia. While the study concluded that BTZ did not improve outcome overall, we examined patient subgroups benefitting from BTZ-containing chemotherapy using proteomic analyses. The proteasome inhibitor BTZ disrupts protein homeostasis and activates cytoprotective heat shock responses. We measured total heat shock factor 1 (HSF1) and phosphorylated HSF1 (HSF1-pSer326) in leukemic cells from 483 pediatric patients using Reverse Phase Protein Arrays. HSF1-pSer326 phosphorylation was significantly lower in pediatric AML compared to CD34+ non-malignant cells. We identified a strong correlation between HSF1-pSer326 expression and BTZ sensitivity. BTZ significantly improved outcome of patients with low-HSF1-pSer326 with a 5-year event-free survival of 44% (ADE) vs. 67% for low-HSF1-pSer326 treated with ADEB (P=0.019). To determine the effect of HSF1 expression on BTZ potency in vitro, cell viability with HSF1 gene variants that mimicked phosphorylated (S326A) and non-phosphorylated (S326E) HSF1-pSer326 were examined. Those with increased HSF1 phosphorylation showed clear resistance to BTZ vs. those with wild type or reduced HSF1-phosphorylation. We hypothesize that HSF1-pSer326 expression could identify patients that benefit from BTZ-containing chemotherapy

Clinical relevance of proteomic profiling in de novo pediatric acute myeloid leukemia:a Children’s Oncology Group study

Pediatric acute myeloid leukemia (AML) remains a fatal disease for at least 30% of patients, stressing the need for improved therapies and better risk stratification. As proteins are the unifying feature of (epi)genetic and environmental alterations, and are often targeted by novel chemotherapeutic agents, we studied the proteomic landscape of pediatric AML. Protein expression and activation levels were measured in 500 bulk leukemic patients’ samples and 30 control CD34(+) cell samples, using reverse phase protein arrays with 296 strictly validated antibodies. The multistep MetaGalaxy analysis methodology was applied and identified nine protein expression signatures (PrSIG), based on strong recurrent protein expression patterns. PrSIG were associated with cytogenetics and mutational state, and with favorable or unfavorable prognosis. Analysis based on treatment (i.e., ADE vs. ADE plus bortezomib) identified three PrSIG that did better with ADE plus bortezomib than with ADE alone. When PrSIG were studied in the context of cytogenetic risk groups, PrSIG were independently prognostic after multivariate analysis, suggesting a potential value for proteomics in combination with current classification systems. Proteins with universally increased (n=7) or decreased (n=17) expression were observed across PrSIG. Certain proteins significantly differentially expressed from normal could be identified, forming a hypothetical platform for personalized medicine

Chromatin Profiles Are Prognostic of Clinical Response to Bortezomib-Containing Chemotherapy in Pediatric Acute Myeloid Leukemia: Results from the COG AAML1031 Trial

The addition of the proteasome inhibitor bortezomib to standard chemotherapy did not improve survival in pediatric acute myeloid leukemia (AML) when all patients were analyzed as a group in the Children\u27s Oncology Group phase 3 trial AAML1031 (NCT01371981). Proteasome inhibition influences the chromatin landscape and proteostasis, and we hypothesized that baseline proteomic analysis of histone- and chromatin-modifying enzymes (HMEs) would identify AML subgroups that benefitted from bortezomib addition. A proteomic profile of 483 patients treated with AAML1031 chemotherapy was generated using a reverse-phase protein array. A relatively high expression of 16 HME was associated with lower EFS and higher 3-year relapse risk after AML standard treatment compared to low expressions (52% vs. 29%, p = 0.005). The high-HME profile correlated with more transposase-accessible chromatin, as demonstrated via ATAC-sequencing, and the bortezomib addition improved the 3-year overall survival compared with standard therapy (62% vs. 75%, p = 0.033). These data suggest that there are pediatric AML populations that respond well to bortezomib-containing chemotherapy