Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests

Many decisions in medicine involve trade-offs, such as between diagnosing patients with disease versus unnecessary additional testing for those who are healthy. Net benefit is an increasingly reported decision analytic measure that puts benefits and harms on the same scale. This is achieved by specifying an exchange rate, a clinical judgment of the relative value of benefits (such as detecting a cancer) and harms (such as unnecessary biopsy) associated with models, markers, and tests. The exchange rate can be derived by asking simple questions, such as the maximum number of patients a doctor would recommend for biopsy to find one cancer. As the answers to these sorts of questions are subjective, it is possible to plot net benefit for a range of reasonable exchange rates in a "decision curve." For clinical prediction models, the exchange rate is related to the probability threshold to determine whether a patient is classified as being positive or negative for a disease. Net benefit is useful for determining whether basing clinical decisions on a model, marker, or test would do more good than harm. This is in contrast to traditional measures such as sensitivity, specificity, or area under the curve, which are statistical abstractions not directly informative about clinical value. Recent years have seen an increase in practical applications of net benefit analysis to research data. This is a welcome development, since decision analytic techniques are of particular value when the purpose of a model, marker, or test is to help doctors make better clinical decisions

Who and when should we screen for prostate cancer? Interviews with key opinion leaders

Prostate cancer screening using prostate-specific antigen (PSA) is highly controversial. In this Q & A, Guest Editors for BMC Medicine's 'Spotlight on Prostate Cancer' article collection, Sigrid Carlsson and Andrew Vickers, invite some of the world's key opinion leaders to discuss who, and when, to screen for prostate cancer. In response to the points of view from the invited experts, the Guest Editors summarize the experts' views and give their own personal opinions on PSA screening

Defining a standard set of patient-centered outcomes for men with localized prostate cancer

Background Value-based health care has been proposed as a unifying force to drive improved outcomes and cost containment. Objective To develop a standard set of multidimensional patient-centered health outcomes for tracking, comparing, and improving localized prostate cancer (PCa) treatment value. Design, setting, and participants We convened an international working group of patients, registry experts, urologists, and radiation oncologists to review existing data and practices. Outcome measurements and statistical analysis The group defined a recommended standard set representing who should be tracked, what should be measured and at what time points, and what data are necessary to make meaningful comparisons. Using a modified Delphi method over a series of teleconferences, the group reached consensus for the Standard Set. Results and limitations We recommend that the Standard Set apply to men with newly diagnosed localized PCa treated with active surveillance, surgery, radiation, or other methods. The Standard Set includes acute toxicities occurring within 6 mo of treatment as well as patient-reported outcomes tracked regularly out to 10 yr. Patient-reported domains of urinary incontinence and irritation, bowel symptoms, sexual symptoms, and hormonal symptoms are included, and the recommended measurement tool is the Expanded Prostate Cancer Index Composite Short Form. Disease control outcomes include overall, cause-specific, metastasis-free, and biochemical relapse-free survival. Baseline clinical, pathologic, and comorbidity information is included to improve the interpretability of comparisons. Conclusions We have defined a simple, easily implemented set of outcomes that we believe should be measured in all men with localized PCa as a crucial first step in improving the value of care. Patient summary Measuring, reporting, and comparing identical outcomes across treatments and treatment centers will provide patients and providers with information to make informed treatment decisions. We defined a set of outcomes that we recommend being tracked for every man being treated for localized prostate cancer

Geophysical anomalies and quartz deformation of the Warburton West structure, central Australia

This paper reports geophysical anomalies and intra-crystalline quartz lamellae in drill cores from the Warburton West Basin overlapping the border of South Australia and the Northern Territory. The pre-Upper Carboniferous ~450 × 300 km-large Warburton Basin, north-eastern South Australia, is marked by distinct eastern and western magnetic, gravity and low-velocity seismic tomography anomalies. Quartz grains from arenite core samples contain intra-crystalline lamellae in carbonate–quartz veins and in clastic grains, similar to those reported earlier from arenites, volcanic rocks and granites from the Warburton East Basin. Universal Stage measurements of quartz lamellae in both sub-basins define Miller–Bravais indices of {10–12} and {10–13}. In-situ quartz lamellae occur only in pre-Late Carboniferous rocks whereas lamellae-bearing clastic quartz grains occur in both pre-Late Carboniferous and post-Late Carboniferous rocks — the latter likely redeposited from the pre-Late Carboniferous basement. Quartz lamellae in clastic quartz grains are mostly curved and bent either due to tectonic deformation or to re-deformation of impact-generated planar features during crustal rebound or/and post-impact tectonic deformation. Seismic tomography low-velocity anomalies in both Warburton West Basin and Warburton East Basin suggest fracturing of the crust to depths of more than 20 km. Geophysical modelling of the Cooper Basin, which overlies the eastern Warburton East Basin, suggests existence of a body of high-density (~2.9–3.0 gr/ cm3 ) and high magnetic susceptibility (SI ~ 0.012–0.037) at a depth of ~6–10 km at the centre of the anomalies. In the Warburton West Basin a large magnetic body of SI= 0.030 is modelled below ~10 km, with a large positive gravity anomaly offset to the north of the magnetic anomaly. In both the Warburton East and Warburton West the deep crustal fracturing suggested by the low velocity seismic tomography complicates interpretations of the gravity data. Universal Stage measurements of quartz lamellae suggest presence of both planar deformation features of shock metamorphic derivation and deformed planar lamella. The latter may be attributed either to redeformation of impact-generated lamella, impact rebound deformation or/and post impact tectonic deformation. The magnetic anomalies in the Warburton East and West sub-basins are interpreted in terms of (1) presence of deep seated central mafic bodies; (2) deep crustal fracturing and (3) removal of Devonian and Carboniferous strata associated with rebound of a central uplift consequent on large asteroid impact. Further tests of the Warburton structures require deep crustal seismic transects

Assessing the incremental value of diagnostic and prognostic markers: A review and illustration

Background New markers may improve prediction of diagnostic and prognostic outcomes. We review various measures to quantify the incremental value of markers over standard, readily available characteristics. Methods Widely used traditional measures include the improvement in model fit or in the area under the receiver operating characteristic (ROC) curve (AUC). New measures include the net reclassification index (NRI) and decision-analytic measures, such as the fraction of true-positive classifications penalized for false-positive classifications [net benefit (NB)]. For illustration, we discuss a case study on the presence of residual tumour vs. benign tissue in 544 patients with testicular cancer. We assessed three tumour markers [Alpha-fetoprotein (AFP), Human chorionic gonadotropin (HCG) and Lactate dehydrogenase (LDH)] for their incremental value over currently standard clinical predictors. Results AUC and R 2 values suggested adding continuous LDH and AFP whereas NB only favoured HCG as a potentially promising marker at a clinically defendable decision threshold of 20% risk. The NRI suggested reclassification potential of all three markers. Conclusions The improvement in standard discrimination measures, which focus on finding variables that might be promising across all decision thresholds, may not detect the most informative markers at a specific threshold of particular clinical relevance. When a marker is intended to support decision-making, calculation of the improvement in a decision-analytic measure, such as NB, is preferable over an overall judgment as obtained from the AUC in ROC analysis

Risk-based prostate cancer screening

Context: Widespread mass screening of prostate cancer (PCa) is not recommended because the balance between benefits and harms is still not well established. The achieved mortality reduction comes with considerable harm such as unnecessary biopsies, overdiagnoses, and overtreatment. Therefore, patient stratification with regard to PCa risk and aggressiveness is necessary to identify those men who are at risk and may actually benefit from early detection. Objective: This review critically examines the current evidence regarding risk-based PCa screening. Evidence acquisition: A search of the literature was performed using the Medline database. Further studies were selected based on manual searches of reference lists and review articles. Evidence synthesis: Prostate-specific antigen (PSA) has been shown to be the single most significant predictive factor for identifying men at increased risk of developing PCa. Especially in men with no additional risk factors, PSA alone provides an appropriate marker up to 30 yr into the future. After assessment of an early PSA test, the screening frequency may be determined based on individualized risk. A limited list of additional factors such as age, comorbidity, prostate volume, family history, ethnicity, and previous biopsy status have been identified to modify risk and are important for consideration in routine practice. In men with a known PSA, risk calculators may hold the promise of identifying those who are at increased risk of having PCa and are therefore candidates for biopsy. Conclusions: PSA testing may serve as the foundation for a more risk-based assessment. However, the decision to undergo early PSA testing should be a shared one between the patient and his physician based on information balancing its advantages and disadvantages

Nonlinear modeling was applied thoughtfully for risk prediction: The Prostate Biopsy Collaborative Group

Abstract Objectives We aimed to compare nonlinear modeling methods for handling continuous predictors for reproducibility and transportability of prediction models. Study Design and Setting We analyzed four cohorts of previously unscreened men who underwent prostate biopsy for diagnosing prostate cancer. Continuous predictors of prostate cancer included prostate-specific antigen and prostate volume. The logistic regression models included linear terms, logarithmic terms, fractional polynomials of degree one or two (FP1 and FP2), or restricted cubic splines (RCS) with three or five knots (RCS3 and RCS5). The resulting models were internally validated by bootstrap resampling and externally validated in the cohorts not used at model development. Performance was assessed with the area under the receiver operating characteristic curve (AUC) and the calibration component of the Brier score (CAL). Results At internal validation models with FP2 or RCS5 showed slightly better performance than the other models (typically 0.004 difference in AUC and 0.001 in CAL). At external validation models containing logarithms, FP1, or RCS3 showed better performance (differences 0.01 and 0.002). Conclusion Flexible nonlinear modeling methods led to better model performance at internal validation. However, when application of the model is intended across a wide range of settings, less flexible functions may be more appropriate to maximize external validity

The Added Value of Percentage of Free to Total Prostate-specific Antigen, PCA3, and a Kallikrein Panel to the ERSPC Risk Calculator for Prostate Cancer in Prescreened Men

Background: Prostate-specific antigen (PSA) testing has limited accuracy for the early detection of prostate cancer (PCa). Objective: To assess the value added by percentage of free to total PSA (%fPSA), prostate cancer antigen 3 (PCA3), and a kallikrein panel (4k-panel) to the European Randomised Study of Screening for Prostate Cancer (ERSPC) multivariable prediction models: risk calculator (RC) 4, including transrectal ultrasound, and RC 4 plus digital rectal examination (4+DRE) for prescreened men. Design, setting, and participants: Participants were invited for rescreening between October 2007 and February 2009 within the Dutch part of the ERSPC study. Biopsies were taken in men with a PSA level ≥3.0 ng/ml or a PCA3 score ≥10. Additional analyses of the 4k-panel were done on serum samples. Outcome measurements and statistical analysis: Outcome was defined as PCa detectable by sextant biopsy. Receiver operating characteristic curve and decision curve analyses were performed to compare the predictive capabilities of %fPSA, PCA3, 4k-panel, the ERSPC RCs, and their combinations in logistic regression models. Results and limitations: PCa was detected in 119 of 708 men. The %fPSA did not perform better univariately or added to the RCs compared with the RCs alone. In 202 men with an elevated PSA, the 4k-panel discriminated better than PCA3 when modelled univariately (area under the curve [AUC]: 0.78 vs 0.62; p = 0.01). The multivariable models with PCA3 or the 4k-panel were equivalent (AUC: 0.80 for RC 4+DRE). In the total population, PCA3 discriminated better than the 4k-panel (univariate AUC: 0.63 vs 0.56; p = 0.05). There was no statistically significant difference between the multivariable model with PCA3 (AUC: 0.73) versus the mode